-- Logs begin at Thu 2022-08-04 20:01:44 UTC, end at Thu 2022-08-04 21:17:07 UTC. -- Aug 04 20:40:23.852745 dv-cinder-836027-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 20:40:23.861423 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 20:40:23 2022] *** Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: nodename: dv-cinder-836027-4-iscsi Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: machine: x86_64 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: clock source: unix Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: pcre jit disabled Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: detected number of CPU cores: 4 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: current working directory: / Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: detected binary path: /usr/bin/uwsgi-core Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: your processes number limit is 47680 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: your memory page size is 4096 bytes Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: detected max file descriptor number: 2048 Aug 04 20:40:23.863454 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: lock engine: pthread robust mutexes Aug 04 20:40:23.864883 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: thunder lock: enabled Aug 04 20:40:23.864883 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 20:40:23.864883 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 20:40:23.902873 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: Python main interpreter initialized at 0x5630cf5769c0 Aug 04 20:40:23.902873 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: python threads support enabled Aug 04 20:40:23.902873 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: your server socket listen backlog is limited to 100 connections Aug 04 20:40:23.902873 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: your mercy for graceful operations on workers is 90 seconds Aug 04 20:40:23.903251 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 20:40:23.903328 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: *** Operational MODE: preforking *** Aug 04 20:40:23.903600 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: *** uWSGI is running in multiple interpreter mode *** Aug 04 20:40:23.903600 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: spawned uWSGI master process (pid: 86866) Aug 04 20:40:23.903802 dv-cinder-836027-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 20:40:23.905373 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: spawned uWSGI worker 1 (pid: 86872, cores: 1) Aug 04 20:40:23.905373 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: spawned uWSGI worker 2 (pid: 86873, cores: 1) Aug 04 20:40:23.905373 dv-cinder-836027-4-iscsi devstack@keystone.service[86866]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 20:40:29.462975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 20:40:29.462975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: from cryptography.utils import int_from_bytes Aug 04 20:40:29.463760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 20:40:29.463760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: from cryptography.utils import int_from_bytes Aug 04 20:40:29.470691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 20:40:29.470691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: from cryptography.utils import int_from_bytes Aug 04 20:40:29.471144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 20:40:29.471144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: from cryptography.utils import int_from_bytes Aug 04 20:40:29.519211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 20:40:29.519387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 20:40:29.519591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] command line args: [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 20:40:29.519876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 20:40:29.520060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ================================================================================ {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 20:40:29.520286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 20:40:29.520412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] admin_token = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.520496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 20:40:29.520642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] config_dir = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.520729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] command line args: [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 20:40:29.520796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.520944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] config_source = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.521028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 20:40:29.521167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ================================================================================ {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 20:40:29.521244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] debug = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.521379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] admin_token = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] config_dir = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] config_source = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] debug = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_config_append = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_dir = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.524953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_file = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] insecure_debug = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_config_append = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_dir = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.525974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_file = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.526775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.527955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] publish_errors = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] notification_format = cadf {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.528858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] standard_threads = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] publish_errors = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.529975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] use_journal = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] use_json = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] use_stderr = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] use_syslog = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] standard_threads = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.530961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] strict_password_check = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.531073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.531204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.531293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.531372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] use_eventlog = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.531522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.531598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] use_journal = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.531731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] use_json = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.531837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.531954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] use_stderr = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.532039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.532116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] use_syslog = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.532238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.532321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] watch_log_file = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 20:40:29.532516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.532593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.532754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.532832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.533978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.534948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.535991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.536938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.537714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.538551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.538895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.539898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.541845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.542957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.543971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.544954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.545868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.546982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.547992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.548896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.549918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.550884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.551827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.enabled = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.552998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.553846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.554901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.555939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.556864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.external = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.557942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.password = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] auth.token = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.558908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.external = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.mapped = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.559960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.password = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] auth.token = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.560987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] catalog.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.561888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.562870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.563849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.564870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.565844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.566989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.567955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.568928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.569960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.570946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.571992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.572976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.573938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.574918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.575851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.576980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.577949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.password = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.578973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.579913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.580905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.581852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.582953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.583924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.584054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.584161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.584311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.584456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.584576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.585194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.585350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.585591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.585829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.585962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.586869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.587983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.588951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.589994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] policy.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] receipt.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.590954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] role.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] role.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.591989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.592905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.593984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] revoke.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] role.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.594954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] role.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] role.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] role.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.595967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.596924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.597984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.598862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.599864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.600029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.600105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.600290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.601908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.602909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.603879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.604853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.provider = fernet {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.connection = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.605849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.606978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] trust.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.607927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.608988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.connection = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.609948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.610044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86872) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 20:40:29.610151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.610390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.610629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.610867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.611105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.611352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.611591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.611837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.612074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.612323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.612581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.612827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.613059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.613302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.613537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 20:40:29.613707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=86873) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 20:40:29.639904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.640139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.640390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.642985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.643215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.643439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.649666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.653100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.680845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.681076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.683042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.683280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.735758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.735907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.787174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.787411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.789026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.789262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.854269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.854501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.854728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.854810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.854950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.855832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.881003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.881233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.881562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.882347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.882505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.882635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.882744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.882865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.883639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.894586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.894885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.907830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.908063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.908279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.908415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.908530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.908614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.909225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.909323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.909323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.909461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.909461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.910005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86872) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.910079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.910079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=86873) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 20:40:29.927357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.927571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 20:40:29.927707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.927838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 20:40:29.927965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.928104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 20:40:29.928230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.928422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 20:40:29.929060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.929461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.929726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.929853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.929982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.930110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.930277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.930538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.939663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.939944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.940217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.940375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.940527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.940668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.940797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.941064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.947678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.948579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.952499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.952786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.953072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.953342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.953471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.953626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.953720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.953888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.954934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.955975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.978973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.980478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.980612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.980749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.981971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.982255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.985377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.985662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.985992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.986875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.987155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.987437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.993419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.993703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.995300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.995584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.997860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.998151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.998440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.998741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.999030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.999326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.999609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.999734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:29.999869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:29.999994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.000227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.000550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.000833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.001121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.001389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.001669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.015875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.016167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.018056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.018338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.020127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.022914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.023381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.023682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.026110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.026406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.028107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.030867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.031542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.034423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.034765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.035069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.035360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.035656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.035936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.036213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.037620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.038883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.039176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.039464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.039753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.040028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.053195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.053572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.053955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.054227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.056969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.057254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.057523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.057798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.061149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.062704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.062967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.064739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.065847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.066130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.066297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.066424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.066588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.066716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.066967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.070429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.070709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.071003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.071287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.071565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.079009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.079291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.079574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.079854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.080138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.080459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.080741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.083011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.083291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.083575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.083859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.084143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.084445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.085292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.096534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.099544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.102038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.103100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.105152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.105632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.107144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.107458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.107754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.108082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.110648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.112144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.112459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.112745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.114257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.115964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.119382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.119673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.119957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.120235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.120388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.121886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.124824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.125117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.125437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.125714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.128358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.128660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.128948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.134350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.134653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.134935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.136055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86872) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.136357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.136642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.136921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.137198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.137474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.137750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.138037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.138433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.138714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.138992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.139258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.139528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.139805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86872) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.141817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=86873) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 20:40:30.142091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.142372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.142650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.142924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.143197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.143469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.143764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.144146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.144454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.144730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.144992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.145267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.145537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=86873) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 20:40:30.174556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5630cf5769c0 pid: 86872 (default app) Aug 04 20:40:30.178191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cdf6-98e5-4257-b442-899fc51719e4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:30.178586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cdf6-98e5-4257-b442-899fc51719e4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:30.178858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5cdf6-98e5-4257-b442-899fc51719e4 None None] PATH_INFO: `/v3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:30.181885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5630cf5769c0 pid: 86873 (default app) Aug 04 20:40:30.183291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Thu Aug 4 20:40:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:37.995326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862725-e608-47a7-b9d6-59577db75349 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:37.995864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862725-e608-47a7-b9d6-59577db75349 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:37.995864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862725-e608-47a7-b9d6-59577db75349 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:38.000245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:38.005589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:38.006001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:38.006143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:38.079636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: INFO dbcounter [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] Registered counter for database keystone Aug 04 20:40:38.099190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG oslo_db.sqlalchemy.engines [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86872) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 20:40:38.106214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writer thread running {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 20:40:38.290930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG passlib.handlers.bcrypt [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86872) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 20:40:38.291356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG passlib.handlers.bcrypt [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86872) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 20:40:38.346645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:38.348245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:38.481994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 477 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 20:40:38.625903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40862725-e608-47a7-b9d6-59577db75349 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:38.696722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: INFO dbcounter [None req-40862725-e608-47a7-b9d6-59577db75349 None None] Registered counter for database keystone Aug 04 20:40:38.717588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG oslo_db.sqlalchemy.engines [None req-40862725-e608-47a7-b9d6-59577db75349 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=86873) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 20:40:38.723729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writer thread running {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 20:40:38.844674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40862725-e608-47a7-b9d6-59577db75349 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:38.848992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:38.849634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:38.849917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:38.850709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:38.859041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:38.910474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING keystone.common.rbac_enforcer.enforcer [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 20:40:38.910751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1cd9d-a4ba-4adc-b563-da13ba5dc064 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:38.921257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Aug 4 20:40:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:40:38.927565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:38.943097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7401990-6a60-4c30-bcb4-9fb29e2f9ff1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:38.948023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:38.949158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:38.949447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:38.949725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] PATH_INFO: `/v3/projects/c6dad5af55404a81a0a649f9204e78ff` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:38.951293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] RBAC: Authorizing `identity:get_project(project_id=c6dad5af55404a81a0a649f9204e78ff)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:39.003745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING keystone.common.rbac_enforcer.enforcer [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 20:40:39.004081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fac8a-3f61-49d7-880b-ddce0f80970a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:39.009432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Thu Aug 4 20:40:38 2022] GET /identity/v3/projects/c6dad5af55404a81a0a649f9204e78ff => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:41.025690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096f3c-9138-4243-b452-055fba16e48f None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:41.026200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096f3c-9138-4243-b452-055fba16e48f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:41.026295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096f3c-9138-4243-b452-055fba16e48f None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:41.030368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:41.035639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:41.036067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:41.036204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:41.056139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:41.057722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:41.065707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:41 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 20:40:41.095323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37096f3c-9138-4243-b452-055fba16e48f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:41.107951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37096f3c-9138-4243-b452-055fba16e48f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:41.112076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:41.112721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:41.113004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:41.113275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:41.121365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:41.123135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4e578-78ed-424f-b9a9-5bdfc63803da admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:41.132811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Thu Aug 4 20:40:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:40:41.138217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:41.148074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d618ac3-d1ba-45be-91bc-c7d6f726ffd3 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:41.152527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:41.153168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:41.153454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:41.153731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:41.155104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:41.156991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a543bd0d-ce44-44fd-a59f-0c1f81cc5865 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:41.158559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Thu Aug 4 20:40:41 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:43.213784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a9bb-d519-44de-ab20-5d6db7709123 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.214323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a9bb-d519-44de-ab20-5d6db7709123 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.214323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a9bb-d519-44de-ab20-5d6db7709123 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.215664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:40:43.220816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.221224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.221524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.242278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:43.243476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:43.273308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417df2-a452-46ac-ba48-6bf1d94cac12 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.273952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417df2-a452-46ac-ba48-6bf1d94cac12 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.274260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f417df2-a452-46ac-ba48-6bf1d94cac12 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.275591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:40:43.281126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.281419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.281702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.324602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG passlib.handlers.bcrypt [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=86873) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 20:40:43.325032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG passlib.handlers.bcrypt [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=86873) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 20:40:43.325727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:40:43.349048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.371104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:43.372749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:43.373447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:43.456351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a00d6-edb3-435d-bc15-ff2ad1fd64fc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:43.477160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:40:43.490382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:43.491342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.491625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.491900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.498231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.503336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:43.512901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:43.513830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:43.514635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:40:43.521197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.601451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4ca5a-e98e-4a37-b9df-cb220436227f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:43.603672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:43.604630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:43.623306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e33148-8252-4333-8639-283703fe5bdc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:43.640543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:43.641409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.641690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.641962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.657686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:43.658317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.658603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.658880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.659630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:43.659933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:43.661369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:43.673780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:43.680455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.714644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:43.716276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:43.819731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:40:43.825644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.828824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:43.893143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702236ce-4bd8-490a-b958-19ba0222e6b2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:43.927406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:43.927594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dc4f3-0385-4047-8b2b-350431a40416 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:43.928646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.928933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.929207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.930392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:43.931887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:43.933764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:43.939128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:43.954771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:43.963996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:43.964892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:43.965177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:43.965453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:43.983832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:43.985405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:40:44.003657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:44.004641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:44.005464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:40:44.005464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:40:44.005464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:40:44.005464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:40:44.005464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:40:44.011445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:44.040784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eecce1-a68d-4612-9b71-e9fb5e8f3f83 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:44.066260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:44.075262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:44.075851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:44.076138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:44.076454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:44.077602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:44.079059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:44.080674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:44.086662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:44.111208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf00fd-c9b6-4363-b54f-bd354ede54e9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:44.145611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:44.146235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:44.146533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:44.146892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:44.147696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:44.147987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:44.149420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2b511-2e2a-4d58-8a42-627a94c0511c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:44.167190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:44.168094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:44.189793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbae99-adac-4deb-bf4a-7b21040e859e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:44.226482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:44.227143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:44.227393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:44.227671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:44.228509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.domain.id=default, user.id=477c6d25e4b7443fa6da61ccbe471661) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:44.228822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:44.230305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec490c1-a517-4fe7-8318-5c813baa2e3d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:44.238039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Aug 4 20:40:44 2022] GET /identity/v3/role_assignments?user.id=477c6d25e4b7443fa6da61ccbe471661&scope.domain.id=default&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:46.598160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc0271-7ec8-4123-b3e4-81cc86db7b2c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:46.598657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc0271-7ec8-4123-b3e4-81cc86db7b2c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:46.598728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc0271-7ec8-4123-b3e4-81cc86db7b2c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:46.600807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:46.606355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:46.607208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:46.607689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:46.636438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:46.638070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:46.692788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ff727-3f39-4ca5-baaf-5edbfa0b9d0b None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:46.693090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ff727-3f39-4ca5-baaf-5edbfa0b9d0b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:46.693354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ff727-3f39-4ca5-baaf-5edbfa0b9d0b None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:46.694756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 12/23] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:40:46.699947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:46.700235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:46.700575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:46.736423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 12/24] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:46.738626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:46.739861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:46.767548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:46.912769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:46.917850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:46.955224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:46.972160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae8760-36d5-4d48-90c7-1a16ceaeb047 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.061064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.061731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.062019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.062297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.083494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:47.093915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c4f48a55-765d-4be9-beae-dbea2e99c9fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:47.099191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:47.100739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:47.101491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 13/26] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:40:47.110561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65948-8cbd-4fc1-8f72-6fa6023294c8 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.110850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65948-8cbd-4fc1-8f72-6fa6023294c8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.111134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65948-8cbd-4fc1-8f72-6fa6023294c8 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.112716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 14/27] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:47.118464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65948-8cbd-4fc1-8f72-6fa6023294c8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:47.191340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73991c-e5e3-4e37-ad27-53223e055744 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.226789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.227423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.227707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.227981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.228736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:47.230139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd657bc-384f-4ed0-9d95-c141ed066000 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:47.261745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65948-8cbd-4fc1-8f72-6fa6023294c8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.300209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.300899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.301189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.301471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.302535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:47.302872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:47.305144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81e26a-a092-4ee7-aba4-4b3f279b6297 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:47.307095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:47.324047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:47.325850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197184-bd08-4aad-9882-e8f7ba964b23 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.326256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197184-bd08-4aad-9882-e8f7ba964b23 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.326607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e197184-bd08-4aad-9882-e8f7ba964b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.347209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3e197184-bd08-4aad-9882-e8f7ba964b23 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:47.348657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3e197184-bd08-4aad-9882-e8f7ba964b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:47.432069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 16/29] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:47.433368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76942f1-08a2-45af-8c59-46c2ab452b5c None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.433657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76942f1-08a2-45af-8c59-46c2ab452b5c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.433936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76942f1-08a2-45af-8c59-46c2ab452b5c None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.444820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 17/30] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:47.445533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76942f1-08a2-45af-8c59-46c2ab452b5c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:47.504176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 14/31] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Aug 4 20:40:46 2022] POST /identity/v3/roles => generated 219 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:40:47.505734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.506066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.506351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.526931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:47.528613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:47.608886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 15/32] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:47.609486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76942f1-08a2-45af-8c59-46c2ab452b5c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.609842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:47.648844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:47.675663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:47.703917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.704611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.704897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.705178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.706391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:47.707842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:47.711244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:47 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:47.717029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:47.738419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229741c7-9b8d-40d0-b3c1-4f80ca5ff1ea None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.789255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.790579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.791055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.791960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.793133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:47.795138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:47.817403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Aug 4 20:40:47 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:40:47.820069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:47.840960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe4ad6-c490-4e0c-9ce3-69e3cdb0e415 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.905871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:47.906682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:47.907063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:47.907433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:47.925233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:47.927175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:47.951561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:40:47.952643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:47.953474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:47.954336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:40:47.954336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:40:47.954336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:40:47.954336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:47.954336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 19/35] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:40:47.962752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b94759-280d-4f0d-bcff-0299bfa7afd2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:47.966432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.007869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.009065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.009557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.010255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.011938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.014487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.017041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 17/36] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:48.025168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.045649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:48.109812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e85996-f1a4-43d2-9dd2-aee7c001680c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.153905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.154920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.156657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.157148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.158108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:48.158625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.160069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be62ff0-78d8-4854-85c4-d50c4ecbf6d2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.160786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c1212-aa65-432e-a3d2-18a807de550c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.182645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 20/37] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:48.184379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9513a-163d-4268-801b-fb5c1472b8c9 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.184379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9513a-163d-4268-801b-fb5c1472b8c9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.184673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a9513a-163d-4268-801b-fb5c1472b8c9 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.185918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:48.188211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9513a-163d-4268-801b-fb5c1472b8c9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.201301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.201990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.202279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.202562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] PATH_INFO: `/v3/domains/default/users/477c6d25e4b7443fa6da61ccbe471661/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.204902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=477c6d25e4b7443fa6da61ccbe471661, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.206295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdfe352-994d-47b5-8c88-f42dd5f2b14d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.221467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 18/39] 10.222.0.52 () {58 vars in 1361 bytes} [Thu Aug 4 20:40:48 2022] PUT /identity/v3/domains/default/users/477c6d25e4b7443fa6da61ccbe471661/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:40:48.222729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c1b5-7f24-4de7-9de6-729458c71de5 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.222997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c1b5-7f24-4de7-9de6-729458c71de5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.223271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481c1b5-7f24-4de7-9de6-729458c71de5 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.224734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 19/40] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:48.225984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301be5-7622-49c7-b5fd-4b3bc34e05c1 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.226292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301be5-7622-49c7-b5fd-4b3bc34e05c1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.226541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9301be5-7622-49c7-b5fd-4b3bc34e05c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.249086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c9301be5-7622-49c7-b5fd-4b3bc34e05c1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:48.250352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c9301be5-7622-49c7-b5fd-4b3bc34e05c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:48.281314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:48.323242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a9513a-163d-4268-801b-fb5c1472b8c9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.331839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:48.343650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 20/41] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:48.354636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971959-7156-4cab-aca3-6b863d973013 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.355500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971959-7156-4cab-aca3-6b863d973013 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.356566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971959-7156-4cab-aca3-6b863d973013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.364421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.365065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.365722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.366011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.366710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.368186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.379347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-16971959-7156-4cab-aca3-6b863d973013 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:48.380608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-16971959-7156-4cab-aca3-6b863d973013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:48.455933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 21/42] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:48.457128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16971959-7156-4cab-aca3-6b863d973013 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.511593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 22/43] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:40:48 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:40:48.512591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.575741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:48.602414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16971959-7156-4cab-aca3-6b863d973013 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.646103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.647449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.648019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.648299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.649675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.651375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.653252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 22/44] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:48.657876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f604d-40e3-4b71-84a7-bd91d9c51afc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.663440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:48.706311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.707249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.707705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.707991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.711440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.711440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa026f-f884-4da9-bdc7-7efad6143356 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.732667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Aug 4 20:40:48 2022] POST /identity/v3/roles => generated 223 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:40:48.813138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0570ea5-ff77-49f9-a417-e26aa092615f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:48.852586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:48.853230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:48.853516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:48.853791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:48.854485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:48.855957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db4998-c8dc-4b1e-b21c-63170116f2e1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:48.891827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:40:48 2022] POST /identity/v3/projects => generated 312 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:40:50.279771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b0a12-0cc2-49ec-bcff-13a028f14300 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.280317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b0a12-0cc2-49ec-bcff-13a028f14300 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.280418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b0a12-0cc2-49ec-bcff-13a028f14300 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.281734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:50.287663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.288101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.288517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.295618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:50.308605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:50.309804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:50.376135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:50.400390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b0a12-0cc2-49ec-bcff-13a028f14300 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:50.419928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:50.505882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b0a12-0cc2-49ec-bcff-13a028f14300 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:50.541288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:50.541871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.542158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.542436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.552712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:50.558519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:50.559396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:50.560222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 25/49] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:40:50.565706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:50.666512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a713b09-10fc-46f7-b0f0-a5db439f3d54 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:50.700491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:50.701111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.701397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.701682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.702396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:50.702705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:50.704098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:50.705717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:50.717100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 25/50] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:50.722690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:50.800955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:50.826157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6309ebb-dd20-43ce-b8f5-ddb35af3af40 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:50.859810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:50.860484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.860770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.861049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.862220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:50.863682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b7932-480e-4eb5-a7f4-928bdcad2968 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:50.869888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:50 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:50.871201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:50.887795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f592-30b8-4a6f-9931-2aabadaec430 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.888091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f592-30b8-4a6f-9931-2aabadaec430 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.888410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0f592-30b8-4a6f-9931-2aabadaec430 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.889749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 27/52] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:40:50.896071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:50.897109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:50.897415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:50.916261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:50.917805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:50.977625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35257e7-5f42-47ad-bbef-0950c360e086 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:50.985344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 28/53] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:51.008793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:51.011510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:51.012160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:51.012476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:51.012756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:51.013901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:51.015463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:51.017097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 26/54] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:51.022869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:51.085549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:51.087224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:51.109456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92891523-bc46-40c0-9fca-bb9f6223b39e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:51.125285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ef809-4d3c-4396-93ec-4b2b5efba86b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:51.145645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:51.146299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:51.147869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:51.147869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:51.148978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:51.150553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:51.152253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 29/55] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:51.158293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:51.161894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:51.162771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:51.163442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:51.163726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:51.164556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.domain.id=default, user.id=477c6d25e4b7443fa6da61ccbe471661) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:51.164869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:51.166315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae0876-9868-46c1-8e3b-a27fb92e1741 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:51.174482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 27/56] 10.222.0.52 () {56 vars in 1322 bytes} [Thu Aug 4 20:40:51 2022] GET /identity/v3/role_assignments?user.id=477c6d25e4b7443fa6da61ccbe471661&scope.domain.id=default&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:40:51.265505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49781832-a1a2-49bf-a866-84e512a29ea9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:51.306550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:51.307182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:51.307570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:51.307853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:51.308587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:51.310078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a2477-d76a-4fa0-98e6-107b8ccf4785 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:51.328468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:51.344691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 30/57] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Aug 4 20:40:51 2022] POST /identity/v3/projects => generated 298 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:40:53.357447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fdd47-0b3a-4762-b8bf-8bfff284c037 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:53.357990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fdd47-0b3a-4762-b8bf-8bfff284c037 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:53.358081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fdd47-0b3a-4762-b8bf-8bfff284c037 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:53.359397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 28/58] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:53.364397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:53.364804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:53.364960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:53.383262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:53.384462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:53.450423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 31/59] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:53.473798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fdd47-0b3a-4762-b8bf-8bfff284c037 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:53.574153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fdd47-0b3a-4762-b8bf-8bfff284c037 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:53.599071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:53.607294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:53.607980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:53.608214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:53.608522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:53.609650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:53.611099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eb391-04b8-40f2-94c4-7f09b90d693e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:53.612696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 29/60] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:53.618850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:53.716739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4001627-5210-41eb-904d-75f1925fabb8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:53.741873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:53.750805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:53.751420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:53.751709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:53.751992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:53.752716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:53.754164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921c8f4-8bd5-4686-a6a6-8db6e506506d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:53.780059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 32/61] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:40:53 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:40:56.848285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2a17-3c6c-4fa2-bf1c-e629fa9f7195 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:56.849315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2a17-3c6c-4fa2-bf1c-e629fa9f7195 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:56.851078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b2a17-3c6c-4fa2-bf1c-e629fa9f7195 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:56.853078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 30/62] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:56.858477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:56.859330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:56.859862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:56.879678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:56.881528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:56.976468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 33/63] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:40:57.002296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5d3e-872d-4fe4-a8c7-d4f604a27ed3 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.002905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5d3e-872d-4fe4-a8c7-d4f604a27ed3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.003406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320c5d3e-872d-4fe4-a8c7-d4f604a27ed3 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.005048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 31/64] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:57.006536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.011781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.012358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.012859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.031592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:57.033146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:57.145026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 32/65] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:40:57.169408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.179495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ae115-e2fa-4bea-90ca-7e98d074260d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.216702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.259478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.260142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.260454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.260733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.282255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:57.304040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:57.305293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:57.306836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 34/66] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:40:57.317352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.318297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78ce0e5-182a-4047-8058-985d6dbc5aa8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.356806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.361079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.361942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.362371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.362789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.375945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ef7e84a9-b234-49a2-8c2b-6f2e27b5b400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:57.384063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:57.384958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:40:57.385700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:40:57.388440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 33/67] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:40:57.389874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b2c4-6351-410a-897b-67c90d1e7abb None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.390449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b2c4-6351-410a-897b-67c90d1e7abb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.390927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe2b2c4-6351-410a-897b-67c90d1e7abb None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.392557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 34/68] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:57.399261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fae4-1204-4c92-bad2-ce4adc347e83 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.399595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fae4-1204-4c92-bad2-ce4adc347e83 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.399884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd0fae4-1204-4c92-bad2-ce4adc347e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.419217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-bfd0fae4-1204-4c92-bad2-ce4adc347e83 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:57.420437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bfd0fae4-1204-4c92-bad2-ce4adc347e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:57.454740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6addced9-187f-43fa-82d1-13a635699193 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.493331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.493962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.494252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.494536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.495265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:57.495576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:57.497117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:57.500473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 35/69] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:40:57.502058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a5f2-67a2-408f-bf13-e3043a01a88a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.502975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a5f2-67a2-408f-bf13-e3043a01a88a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.503453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a5f2-67a2-408f-bf13-e3043a01a88a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.505222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 36/70] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:57.507348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a5f2-67a2-408f-bf13-e3043a01a88a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.508619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 35/71] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:57.518635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.558210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.623890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a5f2-67a2-408f-bf13-e3043a01a88a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.630421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aab428-0bfb-4d29-a4a5-cdccbe8fad85 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.640501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.658767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.659349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.659637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.659918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.660661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:57.660968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:57.663049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b27df0-08c8-46f5-9465-fdaed0c52a72 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:57.666219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.666862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.667148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.667427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.671007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 37/72] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:57.672256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823bd6-4745-4016-bf41-cc78f52484c7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.672580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823bd6-4745-4016-bf41-cc78f52484c7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.672864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e823bd6-4745-4016-bf41-cc78f52484c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.682866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:57.684418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:57.686121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 36/73] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:57.691052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.691550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6e823bd6-4745-4016-bf41-cc78f52484c7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:57.692776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6e823bd6-4745-4016-bf41-cc78f52484c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:57.767050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 38/74] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:57.768104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240344-9f7a-4695-87f4-01c70932e68a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.768534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240344-9f7a-4695-87f4-01c70932e68a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.768794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c240344-9f7a-4695-87f4-01c70932e68a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.770050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 39/75] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:40:57.770740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c240344-9f7a-4695-87f4-01c70932e68a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.778106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.791029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f3f88-6b35-4956-9f8a-eafff38d9743 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.827212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.827856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.828140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.828456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.838895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:57.843955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:57.844787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:40:57.845744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 37/76] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:40:57.849186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:57.884904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c240344-9f7a-4695-87f4-01c70932e68a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:57.895133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:57.921418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:57.921991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.922277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.922557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.923744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:57.925225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4640d-829f-40bf-b0a4-77b28cae845d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:57.926833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 40/77] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:57.928501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:57.928501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:57.928834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:57.947755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:40:57.948966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:40:57.967060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f33fec-168c-4d60-9055-7cdd426e8587 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.001875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.002496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.002781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.003058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.016695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 41/78] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:40:58.017534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.036109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.083476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.085737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.117956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84db417-0478-4e1c-8386-b6bd895a35bf None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.124995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.153563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.154318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.154633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.154908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.165698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:40:58.170592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:40:58.171489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:40:58.172241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 42/79] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:40:58.173415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.189726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 38/80] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:40:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:40:58.190578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.245183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.295812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65604b46-5ab5-4ecd-9bed-ac351b549400 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.296889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0efe-2ec6-4e19-9224-cc4389495acf None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.330117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.330737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.331023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.331297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.332006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:58.332313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.333072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.333739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.333854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.334700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.335136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.342742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 43/81] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.342742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.371608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.408841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.410348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.445061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022bc4f8-708d-4fe2-8522-49a2ccabe64b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.462882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.479308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.479951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.480233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.480550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.481694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.483151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.484746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 44/82] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.485501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.516996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 39/83] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:40:58.522483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.585521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.592179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c7f00a-7fcd-4304-845a-68d0d76bc771 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.622402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cfa3e-fd90-48f1-aa72-19671db9e79a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.633156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.633778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.634087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.634362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.635075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:58.635383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.636833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.651461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 45/84] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.652284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.657031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.657689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.657976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.658259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.659444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.660966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.663298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 40/85] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.663906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.685193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.759639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5880fdb-cb20-43c3-bae0-a7f248970b77 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.765834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e1186-6557-4394-8a73-414fadcae4b3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.794526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.795169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.795454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.795738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.796914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.798360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.799958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 46/86] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.800758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.801419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.802457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.802996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.803442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.804388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.807055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.830044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 41/87] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:40:58 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:40:58.830323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.830866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:58.922702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1349dd-e98e-4235-bf93-6ca9020f00c0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.926046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:58.954904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e3c371-97cf-4949-abb9-8a8913fd8f5f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:58.966846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:58.967802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:58.968089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:58.968686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:58.970367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:58.971882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:58.979314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 47/88] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:58.979314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.001603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.002801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.002801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.002801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.003946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.005453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.008100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 42/89] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:59.008531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.112743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:59.122526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3a9c9e-8a88-4795-9bbd-9899b998e801 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:59.147589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f10bd-9f09-422b-941b-77619750cec9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:59.163919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.164617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.164903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.165179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.165981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:59.166290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.167748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.173636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 48/90] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:40:58 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:59.174527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.233784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.234438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.234726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.235006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.260159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:59.325725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.327294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.400820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496dd7fb-7528-40a9-b59e-4c62402e9946 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:59.442055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.443008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.443748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.444384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.445314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=477c6d25e4b7443fa6da61ccbe471661) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:40:59.445763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.446341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 43/91] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:40:59.446990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.448193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.455626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 49/92] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:40:59 2022] GET /identity/v3/role_assignments?user.id=477c6d25e4b7443fa6da61ccbe471661&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:59.456896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.489191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:59.569173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:40:59.676489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb507-6e22-480a-9ccc-902e09ba6454 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:59.706558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01251d6f-952b-4105-b695-48240d56d0de None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:40:59.770539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.771504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.772068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.772612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.782042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:40:59.782709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:40:59.782995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:40:59.783274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] PATH_INFO: `/v3/projects/263c3fadcd454f52a37050078bfadfca` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:40:59.788432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] RBAC: Authorizing `identity:get_project(project_id=263c3fadcd454f52a37050078bfadfca)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.789930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.791976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 50/93] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:40:59 2022] GET /identity/v3/projects/263c3fadcd454f52a37050078bfadfca => generated 312 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:40:59.798182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:40:59.900064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:40:59.901828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:40:59.990687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:00.008805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:00.014326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef5357-f1d1-463c-8027-5c8b4c2fc1d1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:00.097978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:00.098627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:00.098920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:00.099203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:00.100623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=263c3fadcd454f52a37050078bfadfca, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:00.100946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:00.102407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:00.117347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 44/94] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:00.118544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 51/95] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:40:59 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=263c3fadcd454f52a37050078bfadfca&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 259 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:00.123046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:00.260061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a16ab6a-74ac-4539-925a-ada924cc17be None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:00.322609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:00.323543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:00.323969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:00.324417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:00.325774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:00.327474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18060ce-c3a9-4536-b4cf-db8390b36e6d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:00.332467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 45/96] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:00 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:00.335954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:00.495527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a26cf-ea4d-4390-8acd-a962363e5228 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:00.553863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:00.554592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:00.554884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:00.555170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:00.555979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:00.556296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:00.557790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d781f-6df8-4fd5-8ca0-29c5f1d8b8fe None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:00.564003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 52/97] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:00 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=2b817625cc644df9915eaf992419b0c5 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:01.806675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562902-afb0-4cc4-9108-6df699864d3c None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:01.808842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b451a8-6c84-471e-b98f-3c51bc578758 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:01.809612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b451a8-6c84-471e-b98f-3c51bc578758 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:01.810090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562902-afb0-4cc4-9108-6df699864d3c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:01.810562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b451a8-6c84-471e-b98f-3c51bc578758 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:01.811026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b562902-afb0-4cc4-9108-6df699864d3c None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:01.812606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 53/98] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:01.814212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 46/99] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:01.818336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:01.818866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:01.819357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:01.828408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8fe11-e750-425b-abe3-8bb8cb37572a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:01.828987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8fe11-e750-425b-abe3-8bb8cb37572a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:01.829462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8fe11-e750-425b-abe3-8bb8cb37572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:01.860362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:01.860362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:01.869317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-20a8fe11-e750-425b-abe3-8bb8cb37572a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:01.870224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:01.870877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-20a8fe11-e750-425b-abe3-8bb8cb37572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:01.884740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:01.955574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 54/100] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:02.006913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.062295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 47/101] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 20:41:02.063921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f977-10cd-41fb-a244-33dfacbfb7cc None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.064410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f977-10cd-41fb-a244-33dfacbfb7cc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.064865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79f977-10cd-41fb-a244-33dfacbfb7cc None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.066579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 48/102] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:02.074390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.074978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.075455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.128561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:02.130134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:02.138498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e24887-80d0-488c-b1ef-704c25905946 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.189696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.190660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.191085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.191497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.215298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:02.222601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 49/103] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:02.223684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:02.228900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:02.229837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:02.230575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 55/104] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:02.236423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.261033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.320932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.361979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2247957-289d-4474-9ef2-fe4452acd6f8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.401180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa54438-d0a1-44b3-b458-cf9362077365 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.415985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.417136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.417642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.418114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.431089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:02.437082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:02.438035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:02.438821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:02.441364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 50/105] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:02.442487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.455362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.456009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.456294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.456593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.457329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:02.457635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.460369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:02.475634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 56/106] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:02.476894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.562969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891b3254-7ecd-4ee8-b573-f2ec862189e6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.599070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.600005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.600487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.600911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.602399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.604038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:02.606140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 51/107] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:02.606449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.611415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.674972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.709388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17887147-43bd-4800-b174-bdf1cf494673 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.725509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212b5f5-e6b0-400c-9b32-184ce3ffb38f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.759849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.760828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.761379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.761737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.761908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.762672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:02.762807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.763104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.763378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.763793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] PATH_INFO: `/v3/users/d4aac9f24e154bb3abdff571f10e6b71` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.764882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:02.778752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 57/108] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:02.783797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.791677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] RBAC: Authorizing `identity:get_user(user_id=d4aac9f24e154bb3abdff571f10e6b71)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.793985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf0c05-7e18-4985-8683-a767d4b08fec None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:02.796876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 52/109] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/users/d4aac9f24e154bb3abdff571f10e6b71 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:02.803848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ef41-4305-453e-954c-93933cbc09e0 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.804254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ef41-4305-453e-954c-93933cbc09e0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.804697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292ef41-4305-453e-954c-93933cbc09e0 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.806879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 53/110] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:02.809396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ef41-4305-453e-954c-93933cbc09e0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.890871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.913265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35ba7d-ee44-4497-bd79-9d55d6f35a9a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.946078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:02.948853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292ef41-4305-453e-954c-93933cbc09e0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:02.952644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.952853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.953294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.954032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.955167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.957024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:02.988233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 58/111] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:41:02 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:02.989398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:02.993801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:02.994755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:02.995209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:02.995635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:02.997223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:02.999004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b9a3ae-69d1-4466-90b9-d27664fb970d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:03.009459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 54/112] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:03.009459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642abcc-ba0f-4df2-9ac0-04613adb9dff None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.009459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642abcc-ba0f-4df2-9ac0-04613adb9dff None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.009459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642abcc-ba0f-4df2-9ac0-04613adb9dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.034792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4642abcc-ba0f-4df2-9ac0-04613adb9dff None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:03.037600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4642abcc-ba0f-4df2-9ac0-04613adb9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:03.123051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d581e22-87c0-4447-a27d-f86edf0ac875 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.141917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 55/113] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:03.141917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf544-b4ab-4067-871a-b18d79452af9 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.141917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf544-b4ab-4067-871a-b18d79452af9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.141917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaf544-b4ab-4067-871a-b18d79452af9 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.141917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 56/114] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:03.143517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaf544-b4ab-4067-871a-b18d79452af9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.179704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.180906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.180906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.181102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.220500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.247023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.267727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:03.269222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:03.291453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaf544-b4ab-4067-871a-b18d79452af9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.333527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.334414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.334710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.334992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.336230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:03.337760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ef9ac-49a4-45a6-8134-a8436d7232b8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:03.344597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 57/115] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:03.346849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.347474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.347874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.367863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:03.369158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:03.393015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 59/116] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:03.394157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.450418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 58/117] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:03.452386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.506485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.513207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.528396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9676-ea8c-47de-8add-1da3c3df7487 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.566661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.567231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2a869-dd4b-4f85-af02-27ede21f005a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.567347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.567582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.567858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.580779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:03.586757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:03.587824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:03.588793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:03.590751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 60/118] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:03.591547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.609912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.610595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.610881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.611166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/5890e1b139df49a09f8663306e5ab7bd/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.613145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, user_id=5890e1b139df49a09f8663306e5ab7bd, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:03.614597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24d1cb-f418-4113-9711-754b160817f5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 59/119] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:03 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/5890e1b139df49a09f8663306e5ab7bd/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd565e-a108-4e89-9076-312bf1b7776a None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd565e-a108-4e89-9076-312bf1b7776a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd565e-a108-4e89-9076-312bf1b7776a None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 60/120] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:03.638842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd565e-a108-4e89-9076-312bf1b7776a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.736721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02008dc3-a948-4501-b149-ea6d89db77d2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.765827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd565e-a108-4e89-9076-312bf1b7776a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.784936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.792579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.795911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.796217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.797531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.801337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:03.803204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:03.805041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 61/121] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:03.809594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.823791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.825307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.825798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.826303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.829071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:03.840521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-adea4197-ba84-44c3-8ef1-2aa4bbc0d80f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:03.846493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:03.847379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:03.848105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 61/122] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:03.848863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.849153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.849526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.870387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:03.871732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:03.938733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ad0f9-17e9-4de5-b9ba-ab72621609c7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:03.966267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 62/123] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:03.966955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:03.998089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:03.998643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:03.998922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:03.999194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:03.999906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:04.000208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.001658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.012753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 62/124] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.013453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.097325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.126308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b73c4e-9deb-49a4-9409-e23451fadd57 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.131086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33760556-3e47-4189-b2c4-9c3a43558f6b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.133936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.174604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.175237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.175528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.175807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.176663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=d4aac9f24e154bb3abdff571f10e6b71) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:04.177233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.179232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.179855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.180744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.181341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.181807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.182918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:04.183612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.185416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 63/125] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:03 2022] GET /identity/v3/role_assignments?user.id=d4aac9f24e154bb3abdff571f10e6b71&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.185873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.186061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.197304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 63/126] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.198641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.316380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7f55d-0897-4a16-8c44-b02c31d59ded None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.356213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.357251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.357729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.358214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.359861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.361635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.363951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 64/127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.365368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.369616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.390723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf4123-1b81-4757-b757-1dbacc00a10f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.459385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.460467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.460965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.461449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.474865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:04.487820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:04.488773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:04.489509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:04.490944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd2e45a-2d8f-4370-9820-c0629bcb5b7d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.492927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 64/128] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:04.493961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.543955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.568973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.570127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.570605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.571071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.577563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.579388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.581270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 65/129] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.582225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.636643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925f9611-abdb-4e0f-9e31-8ab5c07a123a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.681393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.682050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.682340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.682624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.686453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.687957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.694304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 65/130] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.695205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.732359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.801076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ede36-183f-452d-9d58-a11dd471025b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:04.842469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:04.843029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:04.843322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:04.843601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:04.844322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:04.844654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:04.846049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:04.865035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 66/131] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:04.865732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:04.919665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:04.934835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f439f2a-689f-4690-b357-a497d2477b56 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.033158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1a1dc4-548d-41ee-81c5-6d8e90ded7f2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.060233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.062379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.064227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.064773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] PATH_INFO: `/v3/projects/263c3fadcd454f52a37050078bfadfca` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.066487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] RBAC: Authorizing `identity:get_project(project_id=263c3fadcd454f52a37050078bfadfca)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.070754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.074955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 66/132] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:04 2022] GET /identity/v3/projects/263c3fadcd454f52a37050078bfadfca => generated 312 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:05.079530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.107457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.108500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.110288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.112155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/477c6d25e4b7443fa6da61ccbe471661/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.117596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, user_id=477c6d25e4b7443fa6da61ccbe471661, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.119359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.129899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:05.151690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 67/133] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:04 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/477c6d25e4b7443fa6da61ccbe471661/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:05.152953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.322235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6bb136-fb1d-4445-b58b-00ce0d470560 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.420854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.421448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.421736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.422024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] PATH_INFO: `/v3/projects/263c3fadcd454f52a37050078bfadfca/users/5890e1b139df49a09f8663306e5ab7bd/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.424234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] RBAC: Authorizing `identity:create_grant(project_id=263c3fadcd454f52a37050078bfadfca, user_id=5890e1b139df49a09f8663306e5ab7bd, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.425774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470035b3-726e-4c51-a8a1-7d0a7d41ee10 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.427206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe117da8-d9e6-4223-9dad-645933a56fbc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.450995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 68/134] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:05 2022] PUT /identity/v3/projects/263c3fadcd454f52a37050078bfadfca/users/5890e1b139df49a09f8663306e5ab7bd/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:05.452503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a56d-5f2b-49ec-9911-7023062c7264 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.452854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a56d-5f2b-49ec-9911-7023062c7264 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.453145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002a56d-5f2b-49ec-9911-7023062c7264 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.455257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 69/135] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:05.466148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.466534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.466890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.492861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.493641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.494039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.494392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.495950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.500248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.502281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 67/136] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:05 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 423 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:05.506065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:05.507522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:05.520109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.572922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:05.579572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:05.590726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 70/137] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:05.632879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.716874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b6d7d-2857-4bea-9d59-760e9b838624 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.764448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf0adbb-e33e-426c-807e-de71380bab91 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.769140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.769810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.770099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.770382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.772401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.773960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.776374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 68/138] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:05 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:05.782557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.807214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:05.808567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:05.809107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:05.809606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:05.813984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:05.815687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:05.817808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 71/139] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:05.823940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:05.956809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:05.974037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489722b9-7610-48a5-b13f-e4e10cf7123e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:05.975846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:05.991488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b78a51-cf21-472c-a698-e3b3cf69ec6a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:06.027345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:06.028048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.028366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.028645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/5890e1b139df49a09f8663306e5ab7bd/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.038695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, user_id=5890e1b139df49a09f8663306e5ab7bd, role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:06.040182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558fc6a-2db0-414e-8a5c-c9f636cf5001 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:06.067173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 69/140] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:05 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/5890e1b139df49a09f8663306e5ab7bd/roles/2b817625cc644df9915eaf992419b0c5 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:41:06.088613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:06.089277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.089548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.089829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.090547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:06.092048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e774-fb23-46eb-8787-beb342008013 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:06.143263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 72/141] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:05 2022] POST /identity/v3/users => generated 283 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:06.258853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8561260-1a85-448f-b46c-3fb6f7fb0f5a None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.259153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8561260-1a85-448f-b46c-3fb6f7fb0f5a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.259354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8561260-1a85-448f-b46c-3fb6f7fb0f5a None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.260807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 70/142] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:06.266625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.266935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.267212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.290561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:06.291782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:06.399080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 73/143] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:06.444569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8561260-1a85-448f-b46c-3fb6f7fb0f5a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:06.564642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8561260-1a85-448f-b46c-3fb6f7fb0f5a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:06.606203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:06.606863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.607157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.607439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.619290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:06.624322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:06.625221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:06.626011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 71/144] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:06.631594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:06.680611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:06.761045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fe73a-9362-4762-94bb-4b139a13ea95 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:06.806645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:06.807196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.807484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.807762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.808511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:06.808819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:06.810212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401d227-afa0-4a90-a87b-b4746fa0a7db None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:06.824393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 74/145] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:06.829488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:06.853242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:06.916866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678768-fcc6-40a7-b908-842823853e28 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.917172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678768-fcc6-40a7-b908-842823853e28 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.917430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678768-fcc6-40a7-b908-842823853e28 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.918922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 75/146] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:06.923998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.924289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.924612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:06.955494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315ad40-f686-4282-a192-dd2d882371fb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:06.964443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:06.965764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:06.998059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:06.998904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:06.999206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:06.999491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.000764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.002244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.003908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 72/147] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:06 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:07.012730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.051418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 76/148] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:07.076188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.112888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:07.151577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:07.166420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff11c06-dc75-4ad0-97ed-fba9e32dd965 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.200900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007165b2-be5f-488e-957f-6f71627f6b97 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.206039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.206672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.206956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.207234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.242076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.242735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.243016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.243288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.260474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:07.265874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:07.266791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:07.267585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 77/149] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:07.273271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.322722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.324215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.396049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604352d-54f9-49cb-af8a-ac1297dcab12 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.420745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:07.477138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:07.481651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 73/150] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:07.488882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.503520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.504221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.504536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.504814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.506140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:07.507098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.508582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.530474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 78/151] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:07.539828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.611928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06ffc43-70a7-4bd7-89f6-d6c34cd8636f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.652048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.652659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.652950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.653228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.654443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.655934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.658762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 74/152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:07.665474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:07.748712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:07.813560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad620e36-329e-4ffd-9bf5-707274a4380a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.838587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795367bb-3645-47f6-90b0-1750071ee5bf None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:07.873991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.875068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.875556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.876025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.877045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:07.877763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.880395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.893132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 75/153] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:07.905956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:07.906964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:07.907452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:07.907944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] PATH_INFO: `/v3/users/d4aac9f24e154bb3abdff571f10e6b71` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:07.909473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] RBAC: Authorizing `identity:get_user(user_id=d4aac9f24e154bb3abdff571f10e6b71)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:07.911230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:07.913840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 79/154] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/users/d4aac9f24e154bb3abdff571f10e6b71 => generated 289 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:07.919595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:08.048669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9daf4-d23d-4586-864e-e7d6e309a4e5 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:08.076384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:08.100723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:08.101995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:08.102656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:08.103155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:08.104721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:08.106447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c51067-3e17-4d1d-a7d9-1d6d5de1bf9e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:08.108646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 76/155] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:07 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:08.115726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:08.143681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:08.266874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1996b2-3b3f-4ccd-ab5c-f1903379b291 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:08.313003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:08.314067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:08.314548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:08.315016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/d4aac9f24e154bb3abdff571f10e6b71/roles/b40ecb05b77448cdb3e4f5ae5208d1f2` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:08.317249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, user_id=d4aac9f24e154bb3abdff571f10e6b71, role_id=b40ecb05b77448cdb3e4f5ae5208d1f2)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:08.319685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760e3e7-b235-419c-a5e9-2e0e0abcb672 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:08.336583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 80/156] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:08 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/users/d4aac9f24e154bb3abdff571f10e6b71/roles/b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:08.972370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944321-55b4-493e-b0f9-c63a10bf0709 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:08.973047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944321-55b4-493e-b0f9-c63a10bf0709 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:08.973541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4944321-55b4-493e-b0f9-c63a10bf0709 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:08.975127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 77/157] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:08.980713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:08.981413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:08.981958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.010661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:09.012194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:09.108284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 81/158] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:09.140614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944321-55b4-493e-b0f9-c63a10bf0709 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:09.270275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4944321-55b4-493e-b0f9-c63a10bf0709 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:09.313248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:09.314185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:09.314642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:09.315089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.328136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:09.333905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:09.334846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:09.335601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:09.337698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 78/159] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:09.343648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:09.356441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:09.461806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40718616-039c-466a-a98f-9d1e343a9d67 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:09.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:09.498101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:09.498537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:09.498962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.499872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:09.500323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:09.501902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7b7bb-8805-48e6-82fc-fce50aef6d70 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:09.511408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 82/160] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:09.517194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:09.588946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:09.648440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17af7c-0738-4a53-b080-1f08e92ce5d7 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:09.648771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17af7c-0738-4a53-b080-1f08e92ce5d7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:09.649000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f17af7c-0738-4a53-b080-1f08e92ce5d7 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.650427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 83/161] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:09.665187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deecf03-1182-4197-a50d-7345e3ee3bb3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:09.669554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:09.669841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:09.670117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.698868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:09.701894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:09.749898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:09.750556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:09.750836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:09.751107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:09.752752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:09.756206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:09.760185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 79/162] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:09 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:09.772668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:09.832241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 84/163] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:09.872643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:09.876728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:09.973197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f998af7-9275-4303-8249-3b262de546c6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:10.010329 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:10.030587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae4016-9ca8-4009-bbeb-921cf077e068 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:10.046867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:10.047497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.047775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.048046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.117704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:10.118292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.118573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.118844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.120744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:10.122210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:10.152714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:10.158210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:10.159086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:10.159855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:10.164726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 85/164] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:10.181391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:10.232570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 80/165] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:10.241435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:10.264734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:10.376694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da993225-2dea-4a47-8fa0-2bd50d1e1c40 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:10.415312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:10.416191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.416672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.417243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] PATH_INFO: `/v3/projects/263c3fadcd454f52a37050078bfadfca` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.418597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] RBAC: Authorizing `identity:get_project(project_id=263c3fadcd454f52a37050078bfadfca)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:10.420234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b9ab9-6897-4e62-a494-c6d77e12e2f5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:10.422665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 81/166] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:10 2022] GET /identity/v3/projects/263c3fadcd454f52a37050078bfadfca => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:10.424027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf5f4-4039-40e1-a5d5-43210dfcb66c None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.424482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf5f4-4039-40e1-a5d5-43210dfcb66c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.424912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bf5f4-4039-40e1-a5d5-43210dfcb66c None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.426245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 82/167] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:10.428762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d251f4-9bee-4288-b668-d970bfed77b4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.429296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d251f4-9bee-4288-b668-d970bfed77b4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.429726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d251f4-9bee-4288-b668-d970bfed77b4 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.431098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 83/168] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:10.432077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d251f4-9bee-4288-b668-d970bfed77b4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:10.442205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:10.544724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc584c-5949-42a5-b251-5923e29980c9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:10.572715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:10.608009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d251f4-9bee-4288-b668-d970bfed77b4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:10.616524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:10.617163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.617446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.617724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.618454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:10.618756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:10.620180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775bff3c-5289-40b7-aba7-6f081f9ab6c4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:10.648543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 86/169] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:10.649828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e65095-cb20-49df-84ad-b2075184673a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.650121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e65095-cb20-49df-84ad-b2075184673a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.650399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e65095-cb20-49df-84ad-b2075184673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.661061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:10.661726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.662001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.662275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.663320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=263c3fadcd454f52a37050078bfadfca, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:10.663623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:10.671957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc44e7c-3f7d-40dd-b346-480255a71f1f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:10.683597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 84/170] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:10 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=263c3fadcd454f52a37050078bfadfca&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:10.688065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:10.688366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:10.688651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:10.692466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-28e65095-cb20-49df-84ad-b2075184673a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:10.694921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-28e65095-cb20-49df-84ad-b2075184673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:10.726630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:10.727875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:10.863308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 85/171] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:10.864196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:10.876684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 87/172] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:10.891984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e65095-cb20-49df-84ad-b2075184673a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:10.990699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:10.999150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.033266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c096f811-696a-45e4-b8db-9be98b51f1ad None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.056003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e65095-cb20-49df-84ad-b2075184673a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.097932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.098824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.099898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.100319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.102319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.103047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.103331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.103701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] PATH_INFO: `/v3/users/477c6d25e4b7443fa6da61ccbe471661` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.104925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] RBAC: Authorizing `identity:get_user(user_id=477c6d25e4b7443fa6da61ccbe471661)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.106394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.109360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 86/173] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:10 2022] GET /identity/v3/users/477c6d25e4b7443fa6da61ccbe471661 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:11.110356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.113931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-a5dafb28-80c7-4e88-a49f-3795aa874a0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:11.121132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:11.122281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:11.123238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:11.126184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 88/174] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:11.127917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724ae4e-01e3-4d22-9817-f2717186b1e7 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.132460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724ae4e-01e3-4d22-9817-f2717186b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.132847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4724ae4e-01e3-4d22-9817-f2717186b1e7 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.135217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 89/175] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:11.139751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724ae4e-01e3-4d22-9817-f2717186b1e7 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.244291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c70b182-f3d2-4865-ba3a-5e1b1227ca06 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.278400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4724ae4e-01e3-4d22-9817-f2717186b1e7 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.282530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.283188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.283473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.283750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.294125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.296095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:11.301041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:11.301932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:11.303128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 87/176] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:11.303781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.314199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.328698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.329826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.330284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.330925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.416170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657c97f-b9af-4b50-a9b2-48cfc314ac08 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.425394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.426915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a309c-d9a8-4b21-9f49-0d07c1a92c3f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.455115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.455750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.456071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.456384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.457108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:11.457466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.459380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5bb67-597e-4e10-afae-ccdab2e6b726 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.469662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 88/177] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:11.470776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.471068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.471354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.491720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:11.493447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:11.546444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 90/178] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:11.548381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d2180-3802-4775-a7df-8e21e9aa3c5d None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.548877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d2180-3802-4775-a7df-8e21e9aa3c5d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.549367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d2180-3802-4775-a7df-8e21e9aa3c5d None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.551032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 91/179] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:11.551936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d2180-3802-4775-a7df-8e21e9aa3c5d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.571557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.576674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.582642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 89/180] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:11.594358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.668742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d2180-3802-4775-a7df-8e21e9aa3c5d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.705144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.705822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.706111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.706385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.707226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:11.707531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.709206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.718526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 92/181] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:11.719272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8762a-ab42-4a66-a0c2-0d9620d086d1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.721415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.757981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.758625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.758905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.759183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.772995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.774458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62ecec-71a0-46f3-a3ad-c52c9e0bd095 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.780514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 90/182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:11.780514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.780514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.780514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.797444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:11.798621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:11.811957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.835065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69cc06-ae68-4655-b13e-abbf6257dde7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.848182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:11.866298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 91/183] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:11.867135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.870082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:11.870717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:11.870994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:11.871261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:11.872410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:11.873846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:11.875469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 93/184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:11.876213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:11.975585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24f19f-7d18-4f0d-a847-7d74fe38802d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:11.975853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f626-e6f0-413a-891a-26781dcb2ee2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.009414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.009726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.010066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.010349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.010435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.010629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.010716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.010901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] PATH_INFO: `/v3/users/5890e1b139df49a09f8663306e5ab7bd` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.012058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5890e1b139df49a09f8663306e5ab7bd)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.013562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.015170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 94/185] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/users/5890e1b139df49a09f8663306e5ab7bd => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.016473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.026207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-17aaab51-bd35-41ab-8a43-41fd41885e48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:12.031314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:12.032200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:12.032999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:12.033846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.033925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 92/186] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:12.034200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a495d-7bf2-4b3a-9677-71d8458780c7 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.034495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a495d-7bf2-4b3a-9677-71d8458780c7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.034772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a495d-7bf2-4b3a-9677-71d8458780c7 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.036442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 93/187] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:12.036994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a495d-7bf2-4b3a-9677-71d8458780c7 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.088699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.127005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22afc2b-fd57-4a67-aa5a-d882e2c934f6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.143244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228a495d-7bf2-4b3a-9677-71d8458780c7 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.162867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.163476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.163759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.164033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.179059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.179683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.179968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.180247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.181070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=477c6d25e4b7443fa6da61ccbe471661) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:12.181383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.182845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.188891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 94/188] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/role_assignments?user.id=477c6d25e4b7443fa6da61ccbe471661&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.188891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.250884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.252410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.269646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.308077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f31b4c-ccb5-4001-982c-dfc1e576db53 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.319753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.347147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.347795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.348072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.349059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.349753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.351212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.352852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 95/189] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.353607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.384522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 95/190] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:12.384522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.469610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9cc2-7d6d-4b76-9034-982b205d2296 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.487477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee4b9c2-bf8c-4e2c-a904-09fa0eae4d3e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.500443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.503175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.503889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.504174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.504489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.527070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.527697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.528010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.528284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.529034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:12.529344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.530754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463549f-57f3-415b-94f4-f5e3dcec03ac None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.539499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 96/191] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.540848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.541135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.541410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.551614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.559095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:12.560230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:12.575340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.576869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.626325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 97/192] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:12.636634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.689003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 96/193] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:12.689838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.731639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea028c-8cc3-4f9d-9079-562e8cc05f40 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.733897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.765629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.766248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.766536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.766812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.767495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.768966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.770157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:12.790186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 98/194] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:12 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:12.793464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de40b44-a298-401f-825d-705218bb7657 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.797476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.833029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.835069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.835372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.835645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.836980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.838501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.840471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 97/195] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.841199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:12.925840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90e343-871c-4bab-8417-a2f71aec3e3f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.964910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986c07e-1ce4-43c9-b9c0-b00035260ae3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:12.970385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:12.971039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:12.971610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:12.971610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:12.973510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:12.976272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:12.978002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 99/196] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:12.978930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.007750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.008763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.009046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.009319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.017465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.022616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:13.027709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:13.028661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 98/197] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:13.029890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.047523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.117091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146eabe3-d62a-436f-8d69-4d324c98a607 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.165896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.166652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.166999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.167282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.168483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.169464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0f709-8dcc-47a4-ae08-b99a523f20a7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.169931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.172810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 100/198] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:12 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:13.173352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.213161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.213781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.214133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.214338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.215209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=8fb818a9f53d4abc91b93e08a03b3e74) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:13.215654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.217060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.223245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 99/199] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/role_assignments?user.id=8fb818a9f53d4abc91b93e08a03b3e74&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:13.224133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.305835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.315148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f284cd2-48ef-4c7e-9c9f-dbafc5244eff None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.359573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.360717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.361196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.361659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.375534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.446307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bef50-e368-4749-9fb8-77a2e99be436 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.480626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.482138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.525137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.526534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.527331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.527999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.529375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:13.529906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.531514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.542780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 100/200] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:13.543941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.621650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.639239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 101/201] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:13.640035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.702119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a314d-a971-4e76-8da2-f2885ec31304 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.746989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:13.760668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.761316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.761599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.761873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.762678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=5890e1b139df49a09f8663306e5ab7bd) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:13.762989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.764499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.783080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 101/202] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/role_assignments?user.id=5890e1b139df49a09f8663306e5ab7bd&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=2b817625cc644df9915eaf992419b0c5 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:13.788694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:13.922664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4530bad2-eb21-4339-81ae-22c45af628a6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.928946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e4254-24e7-4355-9f63-594f47f7d86a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:13.970994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:13.971664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:13.971946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:13.972221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:13.975468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:13.977025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:13.978743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 102/203] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:13.985591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:14.015792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:14.016830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:14.017316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:14.017814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] PATH_INFO: `/v3/users/d4aac9f24e154bb3abdff571f10e6b71` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:14.019231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] RBAC: Authorizing `identity:get_user(user_id=d4aac9f24e154bb3abdff571f10e6b71)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:14.020994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:14.022902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 102/204] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/users/d4aac9f24e154bb3abdff571f10e6b71 => generated 289 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:14.028475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:14.075304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:14.122066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4628a-dec8-43d6-8286-3909391f2579 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:14.163418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:14.164148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:14.165587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:14.165872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:14.166145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:14.166952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=8fb818a9f53d4abc91b93e08a03b3e74) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:14.167273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:14.169752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:14.176109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 103/205] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:13 2022] GET /identity/v3/role_assignments?user.id=8fb818a9f53d4abc91b93e08a03b3e74&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=2b817625cc644df9915eaf992419b0c5 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:14.237075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbde3a-5fe3-4929-a242-a39befa3410f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:14.275134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:14.276036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:14.276488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:14.276898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:14.355036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:14.356871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:14.426816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:14.473236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 103/206] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:14.481493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:14.616793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e0c10-adb1-4912-adbe-2603bd9dbb26 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:14.650494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:14.660430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:14.661255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:14.661700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:14.662113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:14.663434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:14.665051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189bf63-2efe-4945-af92-252c030e15ab None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:14.666836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 104/207] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:14 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:14.672768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:14.776958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbf49c7-b245-428e-84a6-43a4efb75b36 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:14.812419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:14.813352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:14.813817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:14.814293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:14.815219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b, user.id=d4aac9f24e154bb3abdff571f10e6b71) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:14.815703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:14.817357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bcf25-12ac-49bb-ab67-7d9182b7480a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:14.824491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 104/208] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:14 2022] GET /identity/v3/role_assignments?user.id=d4aac9f24e154bb3abdff571f10e6b71&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:15.429974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5af2-439d-4c84-b5fa-2e77cd386bff None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.430257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5af2-439d-4c84-b5fa-2e77cd386bff None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.430533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a5af2-439d-4c84-b5fa-2e77cd386bff None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.431908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 105/209] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:15.443311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833648-4748-4558-a1ed-e32454fa753b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.443311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833648-4748-4558-a1ed-e32454fa753b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.443311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66833648-4748-4558-a1ed-e32454fa753b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.460140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-66833648-4748-4558-a1ed-e32454fa753b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:15.461356 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-66833648-4748-4558-a1ed-e32454fa753b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:15.488034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:15.500303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cefa1-3b25-46da-88c0-9fa22e017c8f None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.500808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cefa1-3b25-46da-88c0-9fa22e017c8f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.500933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cefa1-3b25-46da-88c0-9fa22e017c8f None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.502401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 106/210] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:15.508153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.508502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.508850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.537733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:15.539395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:15.555267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 105/211] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:15.577520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66833648-4748-4558-a1ed-e32454fa753b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:15.613044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 107/212] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:15.639199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:15.684924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66833648-4748-4558-a1ed-e32454fa753b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:15.723098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:15.724400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.724747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.725018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.736590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:15.741674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:15.742557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:15.743309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 106/213] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:15.748593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:15.767539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d3bf9-89ef-4b7b-8aab-da38cca4b080 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:15.769236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:15.810291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:15.821981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:15.822736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.823014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.823284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.824559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:15.826010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4609013-36df-4fea-b97d-5a783a16a41a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:15.827734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 108/214] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:15.828717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef51-36f2-4f27-a05d-d149f59de1ad None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.828993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef51-36f2-4f27-a05d-d149f59de1ad None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.829261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de3ef51-36f2-4f27-a05d-d149f59de1ad None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.830467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 109/215] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:15.832854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ef51-36f2-4f27-a05d-d149f59de1ad None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:15.868055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af09e5-4fa8-4aa1-b136-09c2bda91f63 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:15.907317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:15.908172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.908640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.908916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.909630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:15.909934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:15.911342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f98ecd-0877-4316-8e81-9735f41ad14c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:15.921663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 107/216] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:15.922683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335d95-d325-427e-915d-511e76199eff None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.922971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335d95-d325-427e-915d-511e76199eff None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.923250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f335d95-d325-427e-915d-511e76199eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.944101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6f335d95-d325-427e-915d-511e76199eff None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:15.945290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6f335d95-d325-427e-915d-511e76199eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:15.949772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de3ef51-36f2-4f27-a05d-d149f59de1ad None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:15.989206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:15.990575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:15.991106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:15.991580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:15.992841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:15.995120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.018458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 108/217] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:16.019840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f335d95-d325-427e-915d-511e76199eff None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.020096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 110/218] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:15 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:16.034963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.042118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.107228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.138918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f335d95-d325-427e-915d-511e76199eff None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.176758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a96ad5e-5db5-4345-8156-0bcde2981628 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.184842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.185577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.185866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.186140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] PATH_INFO: `/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.204840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] RBAC: Authorizing `identity:get_user(user_id=fbdc1517c0b94b07bf1e4ef3dbf43f6e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.206308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.212388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 109/219] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:16.214517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.220081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.220890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.221285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.221567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.234282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:16.239667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:16.241033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:16.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:16.242400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 111/220] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:16.248328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.298473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.342883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be112c-b011-435f-b38a-32791f4a68eb None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.387962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac24b7-f336-4e5d-ae19-22744185fc8c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.388584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.389621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.390110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.390586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.403535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.441587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.442228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.442519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.443518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.444408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:16.444724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.446394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.459227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 112/221] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:16.465291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.490583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.492284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.592721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.625197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee02f-305d-4648-8b02-336d15e3f0c9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.668249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.669297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.669787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.670259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.671252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 110/222] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:16.671948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.673649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.675717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 113/223] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:16.680608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.682550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.747000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.785462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c568cce-d8e8-4703-8b24-8bc5428b57c1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.821370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62ee74e-67ca-46c5-a8f6-d6a58a271ba8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:16.829722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.830657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.831086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.831609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.833104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.835428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c92a2-b76b-47b8-885d-e12ab9903c48 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.837966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 111/224] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:16.839410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd028a3-1ec2-47c8-bd9e-ad806f172e04 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.840185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd028a3-1ec2-47c8-bd9e-ad806f172e04 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.840983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd028a3-1ec2-47c8-bd9e-ad806f172e04 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.843405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 112/225] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:16.844638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd028a3-1ec2-47c8-bd9e-ad806f172e04 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:16.872115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:16.872115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.872115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.872115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.872575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:16.872575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dea063-ec32-4050-9f42-c321fc4c40d8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:16.877817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 114/226] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:16.877817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:16.877817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:16.877817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:16.894720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:16.896928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:16.908145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:16.992197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 115/227] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:16.993265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.007516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd028a3-1ec2-47c8-bd9e-ad806f172e04 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.041571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:17.048050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.049216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.049695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.050143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.051092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=fbdc1517c0b94b07bf1e4ef3dbf43f6e) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:17.051549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:17.054352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:17.061471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 113/228] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:16 2022] GET /identity/v3/role_assignments?user.id=fbdc1517c0b94b07bf1e4ef3dbf43f6e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:17.062562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.125597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd24cf-209d-4247-8740-1995e3461cb2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.161522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.162225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.162495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.162776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/users/8fb818a9f53d4abc91b93e08a03b3e74/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.164987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, user_id=8fb818a9f53d4abc91b93e08a03b3e74, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:17.166501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:17.184142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 116/229] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:16 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/users/8fb818a9f53d4abc91b93e08a03b3e74/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:17.254388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:17.260461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c3ea8-3d22-4ebe-95b1-d4841f014044 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.310296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.311435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.311930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.312487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.327318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:17.335441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:17.336394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:17.337629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 114/230] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:17.342782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.462558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:17.467799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746430d-47fe-4173-9a6b-4dc7fc44eb4a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.518151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.518796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.519079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.519356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.520088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:17.532129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:17.533229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:17.547964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 117/231] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:17.557408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.680777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aab5d9-b9a9-4dd4-95f1-bc4693cc03fc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.716733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.717381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.717667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.717943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.719127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:17.720622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:17.722453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 115/232] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:17 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:17.727863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.868803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4e1669-0906-4ad4-959a-844ab74cc7b9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:17.904746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:17.905481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:17.905927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:17.906360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:17.907550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:17.909055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b5ba9a-0488-42df-a50f-f6f0eb368e8d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:17.910722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 118/233] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:17 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:17.916576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:17.947181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:18.021316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e4656-eb4e-4c9c-b197-88b7aa70dd19 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:18.055545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:18.056227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:18.056542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:18.056824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/users/8fb818a9f53d4abc91b93e08a03b3e74/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:18.059052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, user_id=8fb818a9f53d4abc91b93e08a03b3e74, role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:18.060534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d0ea-1a79-4fb5-8f37-dc71579c5b5d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:18.076929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 116/234] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:17 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/users/8fb818a9f53d4abc91b93e08a03b3e74/roles/2b817625cc644df9915eaf992419b0c5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:18.850062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a3c4-90d0-4305-a365-dc007a6f7c67 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:18.850649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a3c4-90d0-4305-a365-dc007a6f7c67 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:18.850649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04a3c4-90d0-4305-a365-dc007a6f7c67 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:18.852198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 119/235] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:18.857950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7af758-6840-4bf0-8a75-75678e8fa6e8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:18.858253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7af758-6840-4bf0-8a75-75678e8fa6e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:18.858536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7af758-6840-4bf0-8a75-75678e8fa6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:18.878004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-aa7af758-6840-4bf0-8a75-75678e8fa6e8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:18.879232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-aa7af758-6840-4bf0-8a75-75678e8fa6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:18.951757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 117/236] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:18.975818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a3c4-90d0-4305-a365-dc007a6f7c67 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:18.981005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610cd2-220e-4993-95fd-0753b1e6ad71 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:18.981317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610cd2-220e-4993-95fd-0753b1e6ad71 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:18.981565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d610cd2-220e-4993-95fd-0753b1e6ad71 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:18.983225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 118/237] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:18.988740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:18.989028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:18.989300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.019295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:19.021412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:19.043135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:19.045149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:19.173539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:19.181134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04a3c4-90d0-4305-a365-dc007a6f7c67 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:19.199270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:19.263885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:19.264576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.264861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.265136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.285374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:19.306544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:19.307486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:19.308241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:19.311888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 120/239] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:19.318641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:19.450814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eb518-97af-44d3-8b12-7acf9d79c8b1 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:19.462838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:19.482213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d789ab-e6c1-4a6f-b949-153ff5b9f60a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:19.529979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:19.531018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.532247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.532884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.534550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:19.535249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:19.538552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:19.551811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 121/240] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:19.557378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:19.558323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.559483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:19.559961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.560447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.562205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:19.566728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:19.571872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 120/241] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:19.577662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:19.622368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:19.753695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fceb6c-fb9f-4d4c-83e1-d0914138c0b8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:19.772691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52355a84-859c-4c8a-9264-f4b52ecbe4d1 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:19.804129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:19.804821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.805107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.805381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.806076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:19.807534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999932f7-5097-465d-8629-8537cbf4a819 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:19.812041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:19.812958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.813450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.813916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] PATH_INFO: `/v3/users/1db8fcb82de04598a66cccab4e73411d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.830382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1db8fcb82de04598a66cccab4e73411d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:19.831415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 121/242] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:19 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:41:19.832458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:19.832926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f25ae-83f3-4bef-b4e2-b9c644006466 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.833870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f25ae-83f3-4bef-b4e2-b9c644006466 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.834248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90f25ae-83f3-4bef-b4e2-b9c644006466 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.835625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 122/243] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:19.836773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 122/244] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:19 2022] GET /identity/v3/users/1db8fcb82de04598a66cccab4e73411d => generated 297 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:19.845178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:19.845178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:19.845178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:19.847010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:19.867901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:19.869347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:19.869888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:19.982526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 123/245] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:20.021476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.023598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.026907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b57607-7a5a-495e-893c-258eb3e866b3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.069472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.071547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.072057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.072554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.156506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de39c71-5c8c-43d0-8f71-c264e43a86c6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.166922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:20.168880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:20.185592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.207253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.207986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.208560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.208922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.222289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-4c694c91-7b50-483d-b734-a694d745eae4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:20.229350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:20.234169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 124/246] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:20.235048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad81cb-a2a5-40e7-a42c-e244557becb0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.237189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad81cb-a2a5-40e7-a42c-e244557becb0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.237189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad81cb-a2a5-40e7-a42c-e244557becb0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.237189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 125/247] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:20.240508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.240932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.241223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.263590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:20.265261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:20.321778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 123/248] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:20.322664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.355810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.371541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 126/249] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:20.372439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.482767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fdd6-ed51-48fb-9d6b-7a1c01edabb3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.506489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.555862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff4f11-acee-490d-821a-4269ac1a9d12 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.563536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.564171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.564483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.564762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.565490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:20.565795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:20.567198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:20.592003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 124/250] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:20.592812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.606924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.607579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.607861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.608151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.609380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:20.610858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:20.612798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 127/251] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:20.613342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.680055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.732371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aa3604-9d4c-4239-9134-48d3b0a95b01 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.750089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee71e74-3e3f-414d-bd33-73364ba53850 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.769125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.770148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.770923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.771328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] PATH_INFO: `/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.772793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] RBAC: Authorizing `identity:get_user(user_id=fbdc1517c0b94b07bf1e4ef3dbf43f6e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:20.774562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:20.776197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 128/252] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:20.776989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.801949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.803282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.803746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.804129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.820238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4d03208d-6083-4054-854c-8f07a7be3e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:20.827323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:20.828190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:20.828979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:20.829893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 125/253] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:20.831044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811ee3-d420-4648-af47-08064ed9dfa2 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.831379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811ee3-d420-4648-af47-08064ed9dfa2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.831661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc811ee3-d420-4648-af47-08064ed9dfa2 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.833164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 126/254] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:20.837167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc811ee3-d420-4648-af47-08064ed9dfa2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.881059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.895652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d89ee-179b-41a5-b94e-55a9de35a948 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:20.926512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:20.936916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:20.937542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:20.937887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:20.938165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:20.938960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=1db8fcb82de04598a66cccab4e73411d) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:20.939268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:20.940851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:20.952681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 129/255] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/role_assignments?user.id=1db8fcb82de04598a66cccab4e73411d&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:20.953856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:20.973776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc811ee3-d420-4648-af47-08064ed9dfa2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.027849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.028548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.028833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.029109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.030728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.032386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed957-513f-4e9a-ba9a-260f09b8ee00 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:21.035381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 127/256] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:21.041173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.041457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.041730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.078802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:21.080040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:21.148580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd102d0-f478-4d20-a884-788cb2c1e9b2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.216592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.217227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.217510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.217786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.218505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:21.218809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.220191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:21.229514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 128/257] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:21.230345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.236117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 130/258] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:21.242256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.275783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:21.354607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473b361-3631-45ac-88df-e6f11e7016c4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.373796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:21.393994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.394738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.395023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.395302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.397403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, user_id=fbdc1517c0b94b07bf1e4ef3dbf43f6e, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.398927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:21.421531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 129/259] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:21 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:21.423182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.457582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9b96a3-83ae-449a-8e86-8480861c0e97 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.550089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:21.558664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.559336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.559786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.560238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.561734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.563421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:21.569829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdb68e-0f37-4d61-9ee5-bac0bb04d9fc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.570313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 131/260] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:21 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:21.589413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.618888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.619855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.620286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.620737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.634712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:21.640323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:21.641513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:21.642281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:21.645108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 130/261] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:21.656992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.769149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd5cd0-8b4d-47b2-a96e-9aa4600809bc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.797306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c9d81-df3d-4def-8030-3cd7234ec77a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:21.806198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:21.822450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.823089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.823375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.823650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.858577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:21.859472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:21.859963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:21.860428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:21.861306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:21.861781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.863462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:21.883789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 131/262] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:21.890515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:21.913178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:21.941416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:21.944063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:22.141659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c35b9d-fcfe-4a6f-aa23-5877f5f61caf None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:22.173924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 132/263] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:22.180927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:22.252695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:22.254140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:22.254829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:22.255864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] PATH_INFO: `/v3/users/8fb818a9f53d4abc91b93e08a03b3e74` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:22.260551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] RBAC: Authorizing `identity:get_user(user_id=8fb818a9f53d4abc91b93e08a03b3e74)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:22.260786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:22.263767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:22.268549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 132/264] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:21 2022] GET /identity/v3/users/8fb818a9f53d4abc91b93e08a03b3e74 => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:22.276931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:22.470172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d8cc-7165-45bf-92bb-378fcccb505d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:22.494716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:22.560265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2121fc79-2d90-402d-83e0-80a136f805ea None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:22.588919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:22.589591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:22.589874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:22.590154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:22.593690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:22.595480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:22.597414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 133/265] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:22 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:22.603999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:22.643491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:22.644467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:22.645348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:22.645923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:22.811045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:22.812608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd80c77-50f8-4e24-aba7-471c9044cd58 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:22.820015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:22.838159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ac323-8112-4516-bff2-57a4a34ab56c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:22.901723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:22.902349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:22.902625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:22.902893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:22.903694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=8fb818a9f53d4abc91b93e08a03b3e74) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:22.903999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:22.905472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:22.918080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 134/266] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:22 2022] GET /identity/v3/role_assignments?user.id=8fb818a9f53d4abc91b93e08a03b3e74&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:22.992961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:23.039939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 133/267] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:23.045477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.108048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1cf08-4062-46ec-af95-2153cbb58cca None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.108253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1cf08-4062-46ec-af95-2153cbb58cca None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.108795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1cf08-4062-46ec-af95-2153cbb58cca None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.110351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 134/268] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:23.124220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870555c2-0be8-459d-bae2-f9da868aeb58 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.124864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870555c2-0be8-459d-bae2-f9da868aeb58 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.125340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870555c2-0be8-459d-bae2-f9da868aeb58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.153572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-870555c2-0be8-459d-bae2-f9da868aeb58 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:23.155165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-870555c2-0be8-459d-bae2-f9da868aeb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:23.219037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c26e0-5378-4c40-9ca6-8a791df80d97 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.251733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 135/269] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:23.253393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418750-70a8-4d84-ba2a-f16b37e66ddc None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.253856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418750-70a8-4d84-ba2a-f16b37e66ddc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.254356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8418750-70a8-4d84-ba2a-f16b37e66ddc None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.256365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 136/270] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:23.269886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.270479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.271027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.274390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.274718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.274998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.275272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.280358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:23.280358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:23.280358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 135/271] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:23.282218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.301044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:23.303688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:23.318758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:23.396911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 137/272] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:23.397750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.401245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18805015-72d3-49cf-8e9a-4cf9f5d4689a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.437239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.438153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.438574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.438985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.440274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:23.442095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:23.448508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 136/273] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:23.450674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.458772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:23.525467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea99ab-e414-4893-974f-2a8c4c81c442 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.556767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:23.562970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.563877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.564163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.564478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.564910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500bda1-b4a1-48b8-ade5-81be2d75005a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.565277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=8fb818a9f53d4abc91b93e08a03b3e74) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:23.565736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:23.567468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:23.574855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 138/274] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/role_assignments?user.id=8fb818a9f53d4abc91b93e08a03b3e74&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=2b817625cc644df9915eaf992419b0c5 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:23.575913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.604898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.605885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.606505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.607070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.607975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:23.609944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:23.637583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 137/275] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:23 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:41:23.726043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fd8dc-3539-4885-bf74-1e5eaf81b49f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.727804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:23.767182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.767849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.768133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.768437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.780848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-088fb67d-baaf-441a-a3d2-b35916f9ecd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:23.786716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:23.787636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:23.788528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:23.789679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 139/276] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:23.797530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.916112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229250a2-7f20-4bc0-8621-af70fa1db686 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:23.955036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:23.955668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.955950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.956226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.956981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:23.957285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:23.959060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78b013-ead2-4f85-bc18-cfccb04fcf9c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:23.965504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5af0-e650-469f-86a2-71786507a25b None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.966139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5af0-e650-469f-86a2-71786507a25b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.966505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ff5af0-e650-469f-86a2-71786507a25b None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.968053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 140/277] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:23.972450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 138/278] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:23.975173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5af0-e650-469f-86a2-71786507a25b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:23.978682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:23.978896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:23.979185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:23.999172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:23.999926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:24.004682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.093159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 139/279] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:24.110154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ff5af0-e650-469f-86a2-71786507a25b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.115807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.155192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.156097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.156420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.156865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] PATH_INFO: `/v3/users/e7584547927d4f4a945f0e45d3f3f012` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.178854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] RBAC: Authorizing `identity:get_user(user_id=e7584547927d4f4a945f0e45d3f3f012)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:24.180372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:24.182120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 141/280] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:23 2022] GET /identity/v3/users/e7584547927d4f4a945f0e45d3f3f012 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:24.189327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.219261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.249799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048052fc-c2a2-4369-af55-15b1fd0ff1ea None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.289924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.290581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.290860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.291193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.298897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.304245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:24.309321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:24.310198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:24.310976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 140/281] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:24.316423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.328834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89747fb-b840-4906-a014-7dc94cb391ef None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.364946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.366035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.366544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.367021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.368046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, user.id=e7584547927d4f4a945f0e45d3f3f012, scope.system=all) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:24.368604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:24.370264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29467c06-de33-4d82-9e1a-a0269e520514 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:24.376231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 142/282] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/role_assignments?user.id=e7584547927d4f4a945f0e45d3f3f012&scope.system=all&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:24.384058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655249de-e7f3-4843-a892-2094cb3bafbd None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.384640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655249de-e7f3-4843-a892-2094cb3bafbd None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.385142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655249de-e7f3-4843-a892-2094cb3bafbd None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.386734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 143/283] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:24.392216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.392842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.393344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.416464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:24.418064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:24.430246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663dc4f-eb2b-4106-9273-a5b1409d0e69 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.469276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.470059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.470346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.470622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.471347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:24.471650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:24.473605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:24.483314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 141/284] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:24.489088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.502371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 144/285] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:24.532919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.553029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.610023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.669045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4285ec7e-b465-401e-aaa5-4a5a68d85c90 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.693030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7716c8-c82b-44c5-a0c4-d5191c09f5c6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.707003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.708039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.708611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.709093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] PATH_INFO: `/v3/users/1db8fcb82de04598a66cccab4e73411d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.710540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] RBAC: Authorizing `identity:get_user(user_id=1db8fcb82de04598a66cccab4e73411d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:24.712185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:24.714839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 142/286] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/users/1db8fcb82de04598a66cccab4e73411d => generated 297 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:24.720652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.776513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.777166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.777442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.777714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.798591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:24.804485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:24.805366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:24.806130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:24.808822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 145/287] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:24.814978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.899972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a138940-ba25-41f5-ad78-07d5100b3462 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:24.934974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:24.946603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:24.947602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:24.948099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:24.948622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:24.950791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:24.952494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:24.954610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 143/288] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:24.961371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:24.977800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:25.048277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b51c96-79ed-4517-85d2-6b5aa482cbcb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:25.100116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0dae6-7e6a-45ad-bb2f-a89fdef196f6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:25.120292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:25.121633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:25.122336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:25.123837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:25.124917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:25.125569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:25.127822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:25.146495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 146/289] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:25.152524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:25.171770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:25.172946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:25.173466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:25.173962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/users/1db8fcb82de04598a66cccab4e73411d/roles/b40ecb05b77448cdb3e4f5ae5208d1f2` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:25.176408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, user_id=1db8fcb82de04598a66cccab4e73411d, role_id=b40ecb05b77448cdb3e4f5ae5208d1f2)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:25.179055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:25.210923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 144/290] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:41:24 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/users/1db8fcb82de04598a66cccab4e73411d/roles/b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:25.269990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e725568a-1f2a-47e2-a955-e377032b8a14 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:25.307521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:25.308101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:25.308413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:25.308702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] PATH_INFO: `/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:25.309913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] RBAC: Authorizing `identity:get_user(user_id=fbdc1517c0b94b07bf1e4ef3dbf43f6e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:25.311397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:25.313108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 147/291] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:25 2022] GET /identity/v3/users/fbdc1517c0b94b07bf1e4ef3dbf43f6e => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:25.318927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:25.429871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:25.482656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85766bf6-d39d-416b-ae5e-735a10aca8da None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:25.529251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:25.529887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:25.530174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:25.530450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:25.616488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:25.617884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:25.691758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:25.749048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 145/292] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:25.754316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:25.763991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:25.867906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70e6e9-c29c-4fb7-92de-df749963d5ef None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:25.907412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1746-090b-42db-b317-603163eb997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:25.908061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1746-090b-42db-b317-603163eb997b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:25.908363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1746-090b-42db-b317-603163eb997b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:25.908632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1746-090b-42db-b317-603163eb997b None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:25.909797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1746-090b-42db-b317-603163eb997b None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:25.911401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1746-090b-42db-b317-603163eb997b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:25.914329 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 148/293] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:25 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:25.919354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:26.076855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c800b-b401-473a-82bc-64d84a84beb0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:26.115531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:26.116162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.116466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.116743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.117534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=fbdc1517c0b94b07bf1e4ef3dbf43f6e) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:26.117841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:26.119455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907ff3c4-d3a5-416b-b0d5-4ab8f39568e0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:26.130818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 146/294] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:25 2022] GET /identity/v3/role_assignments?user.id=fbdc1517c0b94b07bf1e4ef3dbf43f6e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:26.449368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f312fa-45c4-402f-88fc-26e5d721aefd None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.450305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f312fa-45c4-402f-88fc-26e5d721aefd None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.450835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f312fa-45c4-402f-88fc-26e5d721aefd None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.452646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 149/295] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:26.459315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f24ce-61b6-4de5-98cd-7bfcb782a8ea None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.459768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f24ce-61b6-4de5-98cd-7bfcb782a8ea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.459952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f24ce-61b6-4de5-98cd-7bfcb782a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.484476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fa4f24ce-61b6-4de5-98cd-7bfcb782a8ea None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:26.485689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fa4f24ce-61b6-4de5-98cd-7bfcb782a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:26.501807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:26.565322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 147/296] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:26.593531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f312fa-45c4-402f-88fc-26e5d721aefd None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:26.661262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c248-9f10-436c-8dc4-8f2226b21985 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.661530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c248-9f10-436c-8dc4-8f2226b21985 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.661843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c248-9f10-436c-8dc4-8f2226b21985 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.664500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 148/297] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:26.672060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.672648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.673154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.679185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:26.695280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:26.697172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:26.720998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f312fa-45c4-402f-88fc-26e5d721aefd None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:26.765484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:26.766587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.766878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.767158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.772420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 149/298] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:26.780721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:26.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:26.787295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:26.788068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 150/299] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:26.793603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:26.806956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:26.896634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:26.915477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bf0e2-8471-436f-a28d-5e4b9a40b748 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:26.917762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617f59f9-b8b9-40fc-9779-26f08c3e7528 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:26.948594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:26.955164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:26.955808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.956088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.956383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.957431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:26.957559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:26.958774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.959015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.959291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.959563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad0d094-20ad-44ca-ac63-da9947e697c9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:26.959999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:26.960301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:26.961343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 151/300] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:26.962421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a81b-76c8-483b-9dea-f6abd2c0497d None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.962705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a81b-76c8-483b-9dea-f6abd2c0497d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.962976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c5a81b-76c8-483b-9dea-f6abd2c0497d None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:26.964274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 152/301] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:26.967103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d4b55f-a6eb-4ab6-9eab-da5d5f146250 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:26.969252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5a81b-76c8-483b-9dea-f6abd2c0497d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:26.976440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 150/302] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:26.981189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:26.981480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:26.981757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.000423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:27.001700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:27.074067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 151/303] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:27.077364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.092615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c5a81b-76c8-483b-9dea-f6abd2c0497d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.129140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.130273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.130766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.130986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] PATH_INFO: `/v3/groups` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.131605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.133084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.151241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 153/304] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Aug 4 20:41:26 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:27.161440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.163945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.200711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b0085-d0ae-4ac3-95fc-fcf5051e4287 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.249985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.250649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.250930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.251200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] PATH_INFO: `/v3/users/74446506b275453b9aa6dc8b2bbeb68e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.259252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.269989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] RBAC: Authorizing `identity:get_user(user_id=74446506b275453b9aa6dc8b2bbeb68e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.272255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.274548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 152/305] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/users/74446506b275453b9aa6dc8b2bbeb68e => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:27.279974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.329481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707c2441-2de8-4bfa-8dac-63963f1aa3b9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.374864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.375888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.376400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.376900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.398645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:27.405926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:27.406915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:27.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 154/306] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:27.413106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.428478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d3e2d-30ab-4b03-b36d-c2098265f62f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.467683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.477498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.478133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.478419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.478699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.479514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, user.id=74446506b275453b9aa6dc8b2bbeb68e, scope.system=all) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:27.479805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.481858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.496456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 153/307] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/role_assignments?user.id=74446506b275453b9aa6dc8b2bbeb68e&scope.system=all&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:27.553217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a468af-4c57-468a-b7c6-39be12eac0a0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.579129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.595376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.596007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.596332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.596642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.597366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:27.597674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.599689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20f5b-c6d0-4a7e-97e2-e07f89a6dea2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.609751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 155/308] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:27.615437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.713630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571e336-fa5b-41e3-a7b1-dce6e575e50e None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.713951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571e336-fa5b-41e3-a7b1-dce6e575e50e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.714191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571e336-fa5b-41e3-a7b1-dce6e575e50e None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.715543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 156/309] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:27.721787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.722080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.722362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.731992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b43cfd3-6856-44fd-ac15-02d036cd1546 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.746357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:27.747553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:27.769948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.771036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.771522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.771999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] PATH_INFO: `/v3/users/e7584547927d4f4a945f0e45d3f3f012` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.773559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] RBAC: Authorizing `identity:get_user(user_id=e7584547927d4f4a945f0e45d3f3f012)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.775311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.777719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 154/310] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/users/e7584547927d4f4a945f0e45d3f3f012 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:27.783472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.826700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 157/311] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:27.851727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:27.868997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.923276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705957-a16e-4bda-8a3b-1ee9815987f3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.966817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef152a1-cb97-496a-8945-28de03990cb6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:27.969146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:27.970689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:27.970977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:27.971262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] PATH_INFO: `/v3/system/users/e7584547927d4f4a945f0e45d3f3f012/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:27.972866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e7584547927d4f4a945f0e45d3f3f012, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:27.974933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:27.987500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:27.987706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 155/312] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Aug 4 20:41:27 2022] PUT /identity/v3/system/users/e7584547927d4f4a945f0e45d3f3f012/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:41:28.023603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:28.024745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:28.025232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:28.025739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:28.039779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:28.045508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:28.046492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:28.047277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:28.050288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 158/313] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:28.057264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:28.188899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d8df9-0033-43cf-90b5-ee678974a1ce None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:28.218372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:28.228692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:28.229783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:28.230067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:28.230343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:28.231059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:28.231358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:28.232775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:28.242889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 156/314] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:28.248306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:28.363575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ad66e-b7e3-46ed-a2ac-8db5b9338b13 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:28.408042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:28.408728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:28.409013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:28.409288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] PATH_INFO: `/v3/users/1db8fcb82de04598a66cccab4e73411d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:28.410461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=1db8fcb82de04598a66cccab4e73411d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:28.412625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:28.414635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 159/315] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:28 2022] GET /identity/v3/users/1db8fcb82de04598a66cccab4e73411d => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:28.419827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:28.534578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219d4a9-766d-4756-9f7f-28ecc133b3fb None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:28.571041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:28.571662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:28.571942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:28.572217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:28.638355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:28.644182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:28.646522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:28.751087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 157/316] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:28.756694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:28.801496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:28.867047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4598bb9e-2bd6-410e-8125-357853a90ec4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:28.902988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:28.903558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:28.903841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:28.904119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:28.905319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:28.906782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5835c6b-0312-4331-a364-68b941300783 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:28.908757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 160/317] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:28 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:28.914452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:29.004054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:29.017271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d9b63-363f-4942-a407-cd929f5bd396 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:29.055055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:29.055682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.055957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.056224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.057045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, scope.project.id=435212f913ad4e2da779718a72922b5d, user.id=1db8fcb82de04598a66cccab4e73411d) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:29.057350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:29.058777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e63ada-7fc5-4cbc-aa51-70e84c84d419 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:29.065239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 158/318] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:28 2022] GET /identity/v3/role_assignments?user.id=1db8fcb82de04598a66cccab4e73411d&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:29.360895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774201-929b-48f2-881a-a17822d1325d None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.361228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774201-929b-48f2-881a-a17822d1325d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.361455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8774201-929b-48f2-881a-a17822d1325d None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.362947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 161/319] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:29.368475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839471-d635-447a-b9fa-3439fb710799 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.368770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839471-d635-447a-b9fa-3439fb710799 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.369045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f839471-d635-447a-b9fa-3439fb710799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.387858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6f839471-d635-447a-b9fa-3439fb710799 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:29.389060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6f839471-d635-447a-b9fa-3439fb710799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:29.462376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 159/320] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:29.484928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8774201-929b-48f2-881a-a17822d1325d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:29.600531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8774201-929b-48f2-881a-a17822d1325d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:29.610505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:29.637819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579461-c825-423c-ab5e-ade463079adf None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.638305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:29.638418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579461-c825-423c-ab5e-ade463079adf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.638418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579461-c825-423c-ab5e-ade463079adf None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.638542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.638785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.639059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.640372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 160/321] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:29.640501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:29.641652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:29.648297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 162/322] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:29.648452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.648452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.648452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.649158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:29.666943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:29.668173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:29.755786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be9bce-2058-4a5d-8a84-7dd35452fc05 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:29.757950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 161/323] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:29.789447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:29.792843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:29.793463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.793750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.794015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] PATH_INFO: `/v3/groups` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.794666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] RBAC: Authorizing `identity:create_group()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:29.796107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:29.806479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:29.814885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 163/324] 10.222.0.52 () {60 vars in 1134 bytes} [Thu Aug 4 20:41:29 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:41:29.902905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe2e2b-e17f-4176-a9e7-45ce2b1c703f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:29.939135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:29.939790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:29.940073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:29.940364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:29.952570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-6359034b-d771-4e48-84c9-b0aafbcac5e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:29.957620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:29.958806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:29.959571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:29.963806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 162/325] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:29.966034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.006179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:30.080604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd151c-b887-4153-a134-43cdf67b6faa None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.086562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bfdf0-4415-478d-a229-1bb74ea8cacd None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.086845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bfdf0-4415-478d-a229-1bb74ea8cacd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.087118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bfdf0-4415-478d-a229-1bb74ea8cacd None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.088621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 163/326] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:30.093637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.093920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.094186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.120022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:30.123049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:30.144653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.146267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.146728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.147238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.148218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:30.148721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:30.150983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:30.168162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 164/327] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:30.170383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.224634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 164/328] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:30.248622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.251227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:30.299032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:30.303724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44378bf6-71b9-4100-9f60-419fd81df2f0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.360226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.361701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.362555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.363045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] PATH_INFO: `/v3/users/74446506b275453b9aa6dc8b2bbeb68e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.364815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] RBAC: Authorizing `identity:get_user(user_id=74446506b275453b9aa6dc8b2bbeb68e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:30.367092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:30.372979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 165/329] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:30 2022] GET /identity/v3/users/74446506b275453b9aa6dc8b2bbeb68e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:30.375370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.392402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc72aae-6e91-4892-8b8c-010b180ac5f4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.433671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.433671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.434071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.434377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.447239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:30.452209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:30.453123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:30.453892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 165/330] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:30.463460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.512054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc09724-f0d7-476c-b1c0-a372afa5db95 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.556770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:30.564735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.565501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.565857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.566207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] PATH_INFO: `/v3/system/users/74446506b275453b9aa6dc8b2bbeb68e/roles/b40ecb05b77448cdb3e4f5ae5208d1f2` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.568407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=74446506b275453b9aa6dc8b2bbeb68e, role_id=b40ecb05b77448cdb3e4f5ae5208d1f2)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:30.570381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:30.585505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 166/331] 10.222.0.52 () {58 vars in 1334 bytes} [Thu Aug 4 20:41:30 2022] PUT /identity/v3/system/users/74446506b275453b9aa6dc8b2bbeb68e/roles/b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:41:30.597973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440fe-f4f1-4e54-83c2-7edf219f75f6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.643758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.644524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.644726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.644959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.646446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:30.646774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:30.648224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:30.658676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 166/332] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:30.664984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.705263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:30.794869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd5d65-0b81-4d42-baa4-298cf4fd1bae None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.837152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:30.837815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:30.838099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:30.838432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] PATH_INFO: `/v3/users/e7584547927d4f4a945f0e45d3f3f012` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:30.839608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] RBAC: Authorizing `identity:get_user(user_id=e7584547927d4f4a945f0e45d3f3f012)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:30.841085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd2952-b6fb-4ab3-b7ff-cb87f8c30d6f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:30.843185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 167/333] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:30 2022] GET /identity/v3/users/e7584547927d4f4a945f0e45d3f3f012 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:30.849691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:30.962680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373c3e9-1d1b-4285-8c67-927c8b688500 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:30.991963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:31.003899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:31.005337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:31.005848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:31.006187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:31.006989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] RBAC: Adding query filter params (role.id=5aa6a5f83ae944869649edec1f3154eb, user.id=e7584547927d4f4a945f0e45d3f3f012, scope.system=all) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:31.007300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:31.008787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9acddb-58c8-4dc3-9b02-1bc932c3abf9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:31.024540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 167/334] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Aug 4 20:41:30 2022] GET /identity/v3/role_assignments?user.id=e7584547927d4f4a945f0e45d3f3f012&scope.system=all&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:32.214732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c3523-05ea-4db3-9624-4c54c93bc4d3 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:32.215245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c3523-05ea-4db3-9624-4c54c93bc4d3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:32.215319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8c3523-05ea-4db3-9624-4c54c93bc4d3 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:32.216955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 168/335] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:32.234398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:32.234900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:32.234974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:32.256970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:32.258159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:32.346848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 168/336] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:32.390249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c3523-05ea-4db3-9624-4c54c93bc4d3 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:32.525582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:32.534682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8c3523-05ea-4db3-9624-4c54c93bc4d3 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:32.585252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:32.586092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:32.586385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:32.586666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:32.602915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:32.608692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:32.609542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:32.610471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 169/337] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:32.617475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:32.765173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408cdcb-a478-4fd7-829d-c84b80dce5e0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:32.782456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:32.811099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:32.811722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:32.812007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:32.812273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:32.813012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:32.813312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:32.814682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:32.828047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 169/338] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:32.834153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:32.977669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b7922-4781-455f-abde-5c406af1d199 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:33.013817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:33.014454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.014970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.015074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.044645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:33.092002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:33.093612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:33.218183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 170/339] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:33.225484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:33.384045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb04cbb0-3bf6-4d97-b322-adae85465bfd None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:33.435830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:33.436876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.437362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.437826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.445105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:33.446834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d5a87-abd8-4974-914d-0972eb86bfaa None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:33.448767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:33 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:33.454841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:33.529552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410cba2-a9af-443a-b0a9-5efb776f50ac None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.530041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410cba2-a9af-443a-b0a9-5efb776f50ac None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.530311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9410cba2-a9af-443a-b0a9-5efb776f50ac None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.531917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 171/341] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:33.538652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee0c3-b7b9-4001-b59c-31f7dcd993f6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.538937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee0c3-b7b9-4001-b59c-31f7dcd993f6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.539206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee0c3-b7b9-4001-b59c-31f7dcd993f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.557559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:33.581370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-075ee0c3-b7b9-4001-b59c-31f7dcd993f6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:33.582879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-075ee0c3-b7b9-4001-b59c-31f7dcd993f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:33.614694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ed03b-29de-4e0f-ac83-5a249f7ef04c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:33.649897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:33.662893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:33.663800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.664227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.664664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.671749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:33.674135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:33.676724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 171/342] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:33 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:33.683322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:33.730065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 172/343] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:33.769743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58c6a-8a8f-45c4-a348-e4147badc2d9 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.770904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58c6a-8a8f-45c4-a348-e4147badc2d9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.771336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b58c6a-8a8f-45c4-a348-e4147badc2d9 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.773034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 173/344] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:33.778032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5766-8242-4532-bec6-feabaa4d4ede None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.778560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5766-8242-4532-bec6-feabaa4d4ede None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.779019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f5766-8242-4532-bec6-feabaa4d4ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.810224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-301f5766-8242-4532-bec6-feabaa4d4ede None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:33.811937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-301f5766-8242-4532-bec6-feabaa4d4ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:33.843883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228f272-e5c8-476c-9673-766e7c7ab8cd None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:33.874967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:33.899700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:33.900412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.900644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.900924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.901742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:33.902049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:33.903538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:33.914765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 172/345] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:33 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:33.915640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:33.930297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 174/346] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:33.931925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b36b-3a2f-4e41-a5a0-3e14f544be37 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.932476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b36b-3a2f-4e41-a5a0-3e14f544be37 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.932950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de9b36b-3a2f-4e41-a5a0-3e14f544be37 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.934400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 175/347] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:33.940875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:33.941484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:33.941926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:33.982117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:33.983603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:34.133766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d027b-b9d7-407b-88ff-2432b31e5e44 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.140372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 176/348] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:34.141336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.173831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.174412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.174694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.174983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.176964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.193992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-466489f8-e7ed-4515-ac62-65c9df2fa3db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:34.202301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:34.203397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:34.204367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:34.206376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 173/349] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:34.208490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f661a9-f16a-4da4-a35f-c787d12db18e None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.209026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f661a9-f16a-4da4-a35f-c787d12db18e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.210141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f661a9-f16a-4da4-a35f-c787d12db18e None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.212231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 174/350] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:34.213099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f661a9-f16a-4da4-a35f-c787d12db18e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.272056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5731f2a-c559-4000-8b08-356ac67f551e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.282195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.318143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.319215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.319693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.320180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.332021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:34.338107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:34.339020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 20:41:34.339848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:34.342261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 177/351] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:34.343352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.362989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f661a9-f16a-4da4-a35f-c787d12db18e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.409108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.409756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.410038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.410313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.425621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-920feb26-ba6c-4e33-b3e5-52d2647242e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:34.433428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:34.434376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:34.435859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 175/352] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:34.436550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.436550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.436898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.463832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.466257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:34.467848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:34.482477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed3e18-ad24-4100-accc-b629dd4e16c6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.534390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.535339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.536673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.536967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.537710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:34.538018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:34.539456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:34.548454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 178/353] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:34.549363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.568943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 176/354] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:34.570185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.613472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.696576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dbd893-779c-48c1-a490-80c496b0a2ab None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.701438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58da75c-03ff-4dea-89a5-bd85add7c088 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.736375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.737076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.737392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.737671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.738487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:34.738803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:34.741181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:34.747250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.748195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.748734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.749210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.750296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 179/355] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:34.750705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:34.751022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.751409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:34.753063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:34.767704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 177/356] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:34.768954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.773248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.861550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1c846-c4d0-4865-b50e-2bdead9adce9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.878810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:34.884432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e6846-cf17-41c0-8f27-c4588805f643 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:34.897230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.897863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.898149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.898425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.925124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:34.926188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:34.927517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:34.927989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:34.939715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:34.946148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:34.947209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:34.948141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:34.950297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 178/357] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:34.954023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:34.984253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:34.985885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.005813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.084380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33fde-2acb-452a-b84e-a1c49a9ded88 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.123840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.124714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.124930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 180/358] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:35.125113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.125450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] PATH_INFO: `/v3/users/74446506b275453b9aa6dc8b2bbeb68e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.125800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.126808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] RBAC: Authorizing `identity:get_user(user_id=74446506b275453b9aa6dc8b2bbeb68e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.128857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d919dbc-c418-4c35-bd56-e628266d2041 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.132784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 179/359] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:41:34 2022] GET /identity/v3/users/74446506b275453b9aa6dc8b2bbeb68e => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:35.139595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca57eb-dcda-4f2e-bc0f-ba3ef814286f None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.139595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca57eb-dcda-4f2e-bc0f-ba3ef814286f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.139595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca57eb-dcda-4f2e-bc0f-ba3ef814286f None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.139595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 180/360] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:35.139595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca57eb-dcda-4f2e-bc0f-ba3ef814286f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.186254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.281379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7ea60-5716-49d3-86bf-9d6e562546fc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.304415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca57eb-dcda-4f2e-bc0f-ba3ef814286f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.313978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.344164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.348299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.348299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.348299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.352930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.353621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.353903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.354175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.355007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:35.355922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.357466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.371115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 181/361] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:35.372132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.436185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.438280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3796086d-7fa8-43c2-9991-d53ba1afea6c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.500287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.522169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd4cc62-0add-4c84-bdb4-40716e9ba232 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.577771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.578347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.579105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.579105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.581588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.585319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.588774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 182/362] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:35.590209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.598932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 181/363] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:35.600487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.600781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.601053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.621171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:35.622434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:35.635700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.716332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 182/364] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:35.717571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.742560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224628cc-48da-46a0-8d23-22bdeffd9b92 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.780811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.781445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.781729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.782085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.782795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] RBAC: Adding query filter params (role.id=b40ecb05b77448cdb3e4f5ae5208d1f2, user.id=74446506b275453b9aa6dc8b2bbeb68e, scope.system=all) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:35.783269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.784602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.797790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 183/365] 10.222.0.52 () {56 vars in 1308 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/role_assignments?user.id=74446506b275453b9aa6dc8b2bbeb68e&scope.system=all&role.id=b40ecb05b77448cdb3e4f5ae5208d1f2 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:35.798538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.819289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.854795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72100512-d4a2-4575-9e04-1479293b5b74 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.898282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.899294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.899797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.900327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.918399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62281083-fdab-4583-a87f-a83ff69e75e0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:35.921135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:35.959632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:35.960685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:35.961168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:35.961633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:35.963182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.964924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:35.967228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 184/366] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:35.968396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:35.984214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:35.985937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.074519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.101020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2382227-736b-4a30-9e1a-0bbce77b17db None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.111240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 183/367] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:36.112025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.148522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.149310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.149592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.149866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.151316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.153009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.155254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 185/368] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:35 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.156188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.172017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.241405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f8774-0dce-4d57-9691-c45afe6455e6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.280899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9938f9a-3c09-455d-8bfe-7de91eab8658 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.285060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.285860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.286286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.286566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.298507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:36.305745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:36.306670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:36.307475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 184/369] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:36.310142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.326933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.327576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.327838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.328121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.328943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:36.329254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.330689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.337398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 186/370] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=2b817625cc644df9915eaf992419b0c5 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.338065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.358358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.456524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3868ff4c-6b62-4a1d-a844-7f32645a9133 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.466024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.480243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1426-101a-48f3-b697-3afb2c293697 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.506449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.507098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.507374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.507645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.509637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.510295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b5490e-ae6c-469b-8808-b0e42ce1233d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.512148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 185/371] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.513638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bb5a-8aa9-44f6-a832-4c4196d32e4a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.513917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bb5a-8aa9-44f6-a832-4c4196d32e4a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.514188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407bb5a-8aa9-44f6-a832-4c4196d32e4a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.519543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 186/372] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:36.521522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407bb5a-8aa9-44f6-a832-4c4196d32e4a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.532729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.533709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.534298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.534770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.537129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.538838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.541041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 187/373] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.541967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.649111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407bb5a-8aa9-44f6-a832-4c4196d32e4a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.657151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.678392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b83dd-35af-42f5-9d7f-7030a77dab95 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.690477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.691092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.691369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.691637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.693139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:36.693447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.694823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74fb81a-218c-4b4c-9c88-0c7feea26acd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.703683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 187/374] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.705157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.705559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.705845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.726299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:36.727419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:36.729252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.730106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.730586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.730862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.732014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.733594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.735370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 188/375] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.741260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.760453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.810438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 188/376] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:36.811399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.873430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714282-6763-4364-bf92-aba952799dc6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.918124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.918751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.919282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.919560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:36.921031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=435212f913ad4e2da779718a72922b5d) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:36.921556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:36.923642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:36.939012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 189/377] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 260 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:36.939867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:36.941726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848954-e24d-4325-bb95-9ffa151f4413 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:36.960762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:36.985012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:36.986198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:36.986714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:36.987151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.029752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:37.078506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.079978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.097453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ebe2b3-3fc0-4efe-b1be-121a62c37a6c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.144250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.145922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.146712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.149381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.150476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=435212f913ad4e2da779718a72922b5d) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:37.151018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.152711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.159919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 190/378] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=2b817625cc644df9915eaf992419b0c5 => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:37.161932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.229416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 189/379] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:37.240916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.306184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:37.353408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0db4ab-c391-4d6c-ba03-6fa6798b5475 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.375365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:37.412290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.413026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.413310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.413586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.422215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915c9f6-ef69-4acd-b10c-10381b2460d1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.427924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:37.433866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:37.434737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:37.435520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 191/380] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:37.441039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.469216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.470027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.470332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.470695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] PATH_INFO: `/v3/projects/c6dad5af55404a81a0a649f9204e78ff` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.471873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] RBAC: Authorizing `identity:get_project(project_id=c6dad5af55404a81a0a649f9204e78ff)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.474136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.476519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 190/381] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/projects/c6dad5af55404a81a0a649f9204e78ff => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:37.481749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.585095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e30e20-eb70-45ca-b504-48f3286e80ba None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.626767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.627765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.628596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.629067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.630701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] RBAC: Adding query filter params (name=member) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:37.631355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.633001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.639542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:37.644994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 192/382] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:37.651735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.674989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702f7bc3-8dc2-477d-a700-1cf237c6f593 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.752198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.753275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.755359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.757849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] PATH_INFO: `/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.762594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:37.775479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] RBAC: Authorizing `identity:get_group(group_id=b236edb55d4a41c3b3ce263ad9bd116c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.777061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.778948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 191/383] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c => generated 229 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:37.785522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:37.870629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92660db-0676-4223-a297-a6da8cf96607 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:37.918769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:37.919414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:37.919694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:37.919966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:37.922106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:37.925998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:37.928392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 193/384] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:37.937874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:38.014344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b4fdeb-46fa-49dd-9147-6628b424127f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:38.071932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:38.072514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:38.072790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:38.073059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:38.073854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] RBAC: Adding query filter params (group.id=b236edb55d4a41c3b3ce263ad9bd116c, role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=c6dad5af55404a81a0a649f9204e78ff) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:38.074157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:38.075609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:38.084815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:38.097216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 192/385] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/role_assignments?group.id=b236edb55d4a41c3b3ce263ad9bd116c&scope.project.id=c6dad5af55404a81a0a649f9204e78ff&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 260 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:38.128681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa15f676-e195-4f8a-b640-6eecd94189c0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:38.174129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:38.174786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:38.175062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:38.175332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:38.176580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:38.178287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f503f7b-37ed-4b10-8e76-b1305240dc1a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:38.180266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 194/386] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:37 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:38.186060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:38.276748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:38.333649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d0ba4-568c-4cb4-a0bd-6a6c66e757e7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:38.388374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:38.389069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:38.389347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:38.389619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:38.391535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, group_id=ac951f4e7fdb43af87c8556443d44e5e, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:38.393067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc2f0d-982e-4082-83b6-2f4e5c1ef383 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:38.414196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 193/387] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Aug 4 20:41:38 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:38.918504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0f83-f449-469c-ab4c-4334f943c251 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:38.919046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0f83-f449-469c-ab4c-4334f943c251 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:38.919046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c0f83-f449-469c-ab4c-4334f943c251 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:38.920865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 195/388] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:38.926233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a95c-53c1-4bf6-90af-8350fc2e2a2f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:38.926698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a95c-53c1-4bf6-90af-8350fc2e2a2f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:38.926880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331a95c-53c1-4bf6-90af-8350fc2e2a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:38.954277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4331a95c-53c1-4bf6-90af-8350fc2e2a2f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:38.955495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4331a95c-53c1-4bf6-90af-8350fc2e2a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:39.061693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 194/389] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:39.097495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c0f83-f449-469c-ab4c-4334f943c251 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.231484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c0f83-f449-469c-ab4c-4334f943c251 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:39.255679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1a108-bbc5-4562-90ee-72c541669960 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.255950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1a108-bbc5-4562-90ee-72c541669960 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.256500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1a108-bbc5-4562-90ee-72c541669960 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.258172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 195/390] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:39.263072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.263379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.263630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.275890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:39.277032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.278205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.278855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.287203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:39.293464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:39.294941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:39.305919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:39.312541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:39.313507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:39.314280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 196/391] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:39.320367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.379925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:39.417185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 196/392] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:39.456760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.526006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e091-220a-44a9-80fd-fdcac81b0059 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:39.600505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:39.601131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.601414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.601691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.602413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:39.602722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:39.604107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:39.618802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 197/393] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:39.623974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.658478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cde59-ac1e-46c1-9aa4-b48a817aeaea None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:39.730917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:39.731570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.732012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.732468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.732937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:39.748371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:39.751516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390e101-07be-4f5b-8a65-bbc6a2693b5d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-d74d1440-e3b7-4d10-ade6-3ad27abeb686 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:39.753811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:39.755076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:39.755959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:39.758204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 197/394] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:39.759636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e133-93b7-4706-86c4-18e404e98b6c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.760668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e133-93b7-4706-86c4-18e404e98b6c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.761127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18e133-93b7-4706-86c4-18e404e98b6c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.762554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 198/395] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:39.767397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18e133-93b7-4706-86c4-18e404e98b6c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.792862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:39.793413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.793699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.793980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.795165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:39.796748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89ae45-dee2-404d-9662-ba755b6c824e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:39.798683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 198/396] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:39.799932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.800252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.800537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.825770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:39.827369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:39.829066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:39.901714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18e133-93b7-4706-86c4-18e404e98b6c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:39.923423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 199/397] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:39.928882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:39.955092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:39.955903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:39.956282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:39.956710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:39.958105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:39.958685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:39.960492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:39.974578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 199/398] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:39.974578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.013341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.072900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea479efe-67b7-4efe-9ed8-079caf43e1c9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.083779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b5cc2-8198-4c9e-b8e8-d3a5bc745fed None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.087302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.122055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.122789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.123160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.123474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.127657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962556ee-4081-4542-b020-86589ca5022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.128815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962556ee-4081-4542-b020-86589ca5022c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.129393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962556ee-4081-4542-b020-86589ca5022c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.129964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962556ee-4081-4542-b020-86589ca5022c None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.132027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962556ee-4081-4542-b020-86589ca5022c None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.133710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962556ee-4081-4542-b020-86589ca5022c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.134747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:40.136356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 200/399] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:40.137311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962556ee-4081-4542-b020-86589ca5022c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:40.140446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:40.141327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:40.142110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 200/400] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:40.142750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.261548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962556ee-4081-4542-b020-86589ca5022c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.286829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3120f0-bd28-442a-8d88-e78a02b4dd6f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.287140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.299498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.300058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.300350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.300630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.302298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.303541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.309828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 201/401] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:40.309828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.335196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.336211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.336626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.336994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.339713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] RBAC: Authorizing `identity:create_grant(project_id=e5d4b5fb97a74258a6f3faff5b745a1b, group_id=ac951f4e7fdb43af87c8556443d44e5e, role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.342128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.363199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 201/402] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Aug 4 20:41:40 2022] PUT /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/2b817625cc644df9915eaf992419b0c5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:40.367426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.413284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.465605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e2b2d-060f-4868-a0de-1a469b2ab5d7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.509127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.509760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.510038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.510316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.511158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:40.511466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.515709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6a382-b0e3-4691-ba4d-7b4cb6925613 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.521308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f6677-efa7-495f-adcf-f70141df72cd None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.530869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 202/403] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:40.532363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e517af-80ee-432a-b046-93ee8abe40d7 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.532735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e517af-80ee-432a-b046-93ee8abe40d7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.533084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e517af-80ee-432a-b046-93ee8abe40d7 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.539352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 203/404] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:40.539352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e517af-80ee-432a-b046-93ee8abe40d7 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.577127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.577776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.578059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.578336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.580180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.581731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d77ba7-c598-4cb6-bcb2-bc9ea54ec1f6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.584643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 202/405] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:40.585987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.586275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.586474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.605121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:40.606310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:40.614406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.669184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e517af-80ee-432a-b046-93ee8abe40d7 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.697459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 203/406] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:40.698549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.734051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.734802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.735091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.735367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.737520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.739502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.741456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 204/407] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:40.742297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.742749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.850357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdff0cb-fa5f-4994-9386-5aefb35cff3a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.855778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82d45e-3b38-478d-bf83-c51018266a80 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:40.881468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:40.896125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.896966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.897353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.897759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.900725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:40.901994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:40.902422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:40.902839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:40.905405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, group_id=ac951f4e7fdb43af87c8556443d44e5e, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:40.907631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:40.910091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:40.918265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:40.919544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:40.920562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:40.928134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 205/408] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:40.928134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:40.939203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 204/409] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Aug 4 20:41:40 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:40.940293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.045146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.072303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e96eda-8f24-4eb5-8189-2c0de21f9c17 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.081914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89bfc5-76ae-4f28-949f-3f8faedd513f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.117758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.118311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.118592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.118864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.119704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:41.119934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.121332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95bdb8e-7d3c-4ec8-a23d-d957842cc594 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.130680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 205/410] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:41.132084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991a41-0e80-4639-8dd7-61f3de3dd5e8 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.132462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991a41-0e80-4639-8dd7-61f3de3dd5e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.132696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d991a41-0e80-4639-8dd7-61f3de3dd5e8 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.134037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 206/411] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:41.136178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d991a41-0e80-4639-8dd7-61f3de3dd5e8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.139706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.140399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.140691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.141294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.144005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.145778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25598300-ec64-4025-b90c-22a3936f4ef9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.148413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 206/412] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:40 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:41.150402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.150689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.150968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.176733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:41.178193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:41.250783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d991a41-0e80-4639-8dd7-61f3de3dd5e8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.251323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.268662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 207/413] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:41.269651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.312166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.313127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.313415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.313697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] PATH_INFO: `/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.315544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] RBAC: Authorizing `identity:get_group(group_id=b236edb55d4a41c3b3ce263ad9bd116c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.317862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.320762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 207/414] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:41 2022] GET /identity/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:41.321607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.341573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.395509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7965853-cd29-4a23-83c1-fca42654cbfc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.436021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.437186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.437632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.438066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.440537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] RBAC: Authorizing `identity:create_grant(project_id=435212f913ad4e2da779718a72922b5d, group_id=ac951f4e7fdb43af87c8556443d44e5e, role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.442459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.459114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 208/415] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Aug 4 20:41:41 2022] PUT /identity/v3/projects/435212f913ad4e2da779718a72922b5d/groups/ac951f4e7fdb43af87c8556443d44e5e/roles/2b817625cc644df9915eaf992419b0c5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:41.460298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.472467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106ad4e-2c06-42c0-8fbd-2ff3895a0446 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.510966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.543249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.543918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.544209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.544563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.561640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:41.573374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:41.574497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:41.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 208/416] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:41.580682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.599955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49c378-583c-4cf3-87ce-84a018ff0101 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.640188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.641155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.641632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.642052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] PATH_INFO: `/v3/projects/c6dad5af55404a81a0a649f9204e78ff` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.643523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] RBAC: Authorizing `identity:get_project(project_id=c6dad5af55404a81a0a649f9204e78ff)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.645843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.647976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 209/417] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:41 2022] GET /identity/v3/projects/c6dad5af55404a81a0a649f9204e78ff => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:41.655310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.702959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.771946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcdf5b-1d9f-4d19-866e-4936e8de478e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.812665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.813294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.813600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.813929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.814568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:41.814873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.816262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.828397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 209/418] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:41.833737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670527a0-eb77-49e1-91ea-45235366183d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.834844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:41.844876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:41.876123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.877179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.877666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.878145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] PATH_INFO: `/v3/projects/c6dad5af55404a81a0a649f9204e78ff/groups/b236edb55d4a41c3b3ce263ad9bd116c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:41.881607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c6dad5af55404a81a0a649f9204e78ff, group_id=b236edb55d4a41c3b3ce263ad9bd116c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:41.883453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:41.900138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 210/419] 10.222.0.52 () {58 vars in 1442 bytes} [Thu Aug 4 20:41:41 2022] PUT /identity/v3/projects/c6dad5af55404a81a0a649f9204e78ff/groups/b236edb55d4a41c3b3ce263ad9bd116c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:41:41.953062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cf450-c2d9-4228-b9d3-909a578a9ff0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:41.996539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:41.997201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:41.997493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:41.997773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.000875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:42.086621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:42.088123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:42.199667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 210/420] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:42.205220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:42.343232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663f0be-bc8a-4811-b642-b64257c0d1b9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:42.394382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:42.395108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:42.395391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:42.395663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.396913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:42.398351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:42.400387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 211/421] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:42 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:42.405531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:42.467937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:42.571599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0474af2-2fbb-4d21-ada3-03d00fd6b422 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:42.611641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:42.612293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:42.612600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:42.612874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.615126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:42.616658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99955e4c-63be-4a93-9f4b-590f32fd027d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:42.618556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 211/422] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:42 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:42.624852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:42.688940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:42.776274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583d4e-1696-4a80-80de-cad02411b948 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:42.776606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583d4e-1696-4a80-80de-cad02411b948 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:42.776861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff583d4e-1696-4a80-80de-cad02411b948 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.778309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 212/423] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:42.780652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b89c1ae-1abf-4879-8344-b6db94ae2533 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:42.786456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968aac-7122-4510-8ccd-e94e1d6d2dd4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:42.786809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968aac-7122-4510-8ccd-e94e1d6d2dd4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:42.787130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31968aac-7122-4510-8ccd-e94e1d6d2dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.822236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:42.823222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:42.823716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:42.824192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:42.825359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:42.825907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:42.827567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:42.829342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-31968aac-7122-4510-8ccd-e94e1d6d2dd4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:42.830554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-31968aac-7122-4510-8ccd-e94e1d6d2dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:42.836033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 212/424] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:42 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 611 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:42.983020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 213/425] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:43.028856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.070795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440697c-20e5-4c2e-84e1-2a08faea7dc0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.071091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440697c-20e5-4c2e-84e1-2a08faea7dc0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.071339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a440697c-20e5-4c2e-84e1-2a08faea7dc0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.072997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 214/426] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:43.077929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.078203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.078471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.101193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:43.102415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:43.133175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.164165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9504d-763d-4e7f-aea3-b9d9bb46177e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.172712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.192484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 215/427] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:43.205728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.206506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.206900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.207291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.217060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.220099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:43.227754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:43.228686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:43.229668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 213/428] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:43.235322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.347613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77268603-cab0-4bfe-ac93-367ccc99d83c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.368300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c0eb-a60e-4a8a-a81f-cb01a793b66a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.394229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.395765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.396062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.396359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] PATH_INFO: `/v3/roles/member` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.407904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.408850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.409323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:43.409537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.410015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.411251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:43.411796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:43.413416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:43.414604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:43.415459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:43.421215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 216/429] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:43.424530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.431196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 214/430] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:43.436305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.452983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.476878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.574478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae368e5-3819-49a8-8f2a-fe3b618a9384 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.596769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa1996-00e5-4c1a-9eb5-3a11135b28e9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.627404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.628820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.629834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.630491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.631485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] RBAC: Adding query filter params (name=member) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:43.633578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:43.634179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:43.646122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 217/431] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:43.654564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.660316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.661282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.661713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.662135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.747474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:43.749335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:43.767680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.785316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:43.798330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156109a3-21f8-439e-97e5-ccc44363281b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:43.849954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:43.850958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:43.851443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:43.851926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:43.890715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 215/432] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:43.895590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:43.943817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:43.945317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.033186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0593-36e4-4aca-b2f6-13d64aa84f3e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.046348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.067627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.070457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 218/433] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:44.071794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.082874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.084535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.085513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.086012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] PATH_INFO: `/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.087746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] RBAC: Authorizing `identity:get_project(project_id=e5d4b5fb97a74258a6f3faff5b745a1b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.090475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.093364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 216/434] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:43 2022] GET /identity/v3/projects/e5d4b5fb97a74258a6f3faff5b745a1b => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:44.101663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.190745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6225917d-d701-4f55-85cb-1894e4afa81d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.221551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e30eaf-3f46-4a52-ac99-15ccc38a1b97 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.227693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.228648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.229134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.229706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.231039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.232524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49831591-61ff-4bbe-855d-2049b7e5ea6c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.238976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 219/435] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:44.240355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0cde9-b036-48c6-904a-6d0a008bdeb9 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.240645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0cde9-b036-48c6-904a-6d0a008bdeb9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.240988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0cde9-b036-48c6-904a-6d0a008bdeb9 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.244384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 220/436] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:44.244384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0cde9-b036-48c6-904a-6d0a008bdeb9 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.261714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.262357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.262637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.262910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.264278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.265805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcea009-b01f-494f-a1ac-2e3272ab2b39 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.267850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 217/437] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:44.268999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.269282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.269554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.288427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:44.289683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:44.310888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.349326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.363435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 218/438] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:44.364750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.376883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0cde9-b036-48c6-904a-6d0a008bdeb9 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.417099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.417776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.418059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.418340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.419483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.421101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.423131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 221/439] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:44.425805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.476086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06f7f17-28a5-4c87-9801-81310e12b350 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.510635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.511243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.511523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.511793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.512592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:44.512898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.514335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.520324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 219/440] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=e5d4b5fb97a74258a6f3faff5b745a1b&role.id=2b817625cc644df9915eaf992419b0c5 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:44.523412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.543612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155b04a-5318-45ca-a3e5-ab9ca30edf49 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.570295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.585030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.592731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.594374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.594872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.595515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.611053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-992a280b-e1b2-45ce-ba17-2a606960724a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:44.618676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:44.619682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:44.620486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:44.626656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 222/441] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:44.626656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f681-3a9c-463e-b36a-eb9c1dd65d01 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.626656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f681-3a9c-463e-b36a-eb9c1dd65d01 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.626656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411f681-3a9c-463e-b36a-eb9c1dd65d01 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.626656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 223/442] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:44.627369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411f681-3a9c-463e-b36a-eb9c1dd65d01 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.642058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb1a4f-b2ce-4006-937e-2404e5fb6030 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.679653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.680281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.680593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.680870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.681664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=5aa6a5f83ae944869649edec1f3154eb, scope.project.id=435212f913ad4e2da779718a72922b5d) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:44.682445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.683412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551faf7-ad41-4460-9921-55c9842836a8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.690287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 220/443] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=5aa6a5f83ae944869649edec1f3154eb => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:44.693326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.693326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.693326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.713804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:44.715049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:44.765315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411f681-3a9c-463e-b36a-eb9c1dd65d01 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.813775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:44.814154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 221/444] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:44.814780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:44.814988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:44.815288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:44.815981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:44.816285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:44.818394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:44.831008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 224/445] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:44.837080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.840819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:44.894058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.899010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:44.967651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241a3e52-c8bc-4746-8219-17f63ab99dcc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:44.991367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6490d3-b664-448d-910b-3d4118719d55 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.010791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.011649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.011930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.012196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.034057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.034632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.034909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.035183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.046358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:45.053338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:45.054395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:45.055340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 225/446] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:45.062085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.093410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.094830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.153855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.182707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1ee74-b5d8-4f35-b0cc-c0795d6ca723 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.184219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.210345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 222/447] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:45.216035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.217246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.217863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.218104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.218381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.219090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:45.219392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.220817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.230004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 226/448] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.235317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.314862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c38a7-cb39-413b-bbe3-65c649531692 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.349677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add16e92-01dd-40ca-a12c-a3064d5e65c3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.350484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.351133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.351405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.351674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] PATH_INFO: `/v3/projects/435212f913ad4e2da779718a72922b5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.352872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] RBAC: Authorizing `identity:get_project(project_id=435212f913ad4e2da779718a72922b5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.354310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.356113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 223/449] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/projects/435212f913ad4e2da779718a72922b5d => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.361369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.376143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.383806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.384453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.384738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.385010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.416650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.454367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.455838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.462079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0b0cc-d447-4b5c-8d06-52e1d325638f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.496248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.496912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.497190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.497467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] PATH_INFO: `/v3/groups/ac951f4e7fdb43af87c8556443d44e5e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.498620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] RBAC: Authorizing `identity:get_group(group_id=ac951f4e7fdb43af87c8556443d44e5e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.500059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.501700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 224/450] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/groups/ac951f4e7fdb43af87c8556443d44e5e => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.508454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.553754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 227/451] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:41:45.559357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.605772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.618171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537383b-2fd4-4c42-aa05-269dbca64d3d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.629127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:45.653028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.653638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.653919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.654189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.654968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] RBAC: Adding query filter params (group.id=ac951f4e7fdb43af87c8556443d44e5e, role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=435212f913ad4e2da779718a72922b5d) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:45.655272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.656723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.662585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 225/452] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/role_assignments?group.id=ac951f4e7fdb43af87c8556443d44e5e&scope.project.id=435212f913ad4e2da779718a72922b5d&role.id=2b817625cc644df9915eaf992419b0c5 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.672592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415be25a-8aa5-448a-a96e-08b410af5a60 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.711961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.712972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.713403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.713824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] PATH_INFO: `/v3/projects/c6dad5af55404a81a0a649f9204e78ff` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.715362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] RBAC: Authorizing `identity:get_project(project_id=c6dad5af55404a81a0a649f9204e78ff)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.716972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.718734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 228/453] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/projects/c6dad5af55404a81a0a649f9204e78ff => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.724773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.834387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ad04e-92f5-44ef-84f0-f12ff92e236e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:45.868603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:45.869175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:45.869459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:45.869766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] PATH_INFO: `/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:45.870909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] RBAC: Authorizing `identity:get_group(group_id=b236edb55d4a41c3b3ce263ad9bd116c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:45.872403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb50acf3-be99-495d-9064-27cccf2e6358 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:45.874091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 226/454] 10.222.0.52 () {56 vars in 1180 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/groups/b236edb55d4a41c3b3ce263ad9bd116c => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:45.880786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:45.987180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53030e-2997-4d69-ab81-f967406d4602 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:46.019305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:46.024581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:46.025192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:46.025469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:46.025739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:46.026515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] RBAC: Adding query filter params (group.id=b236edb55d4a41c3b3ce263ad9bd116c, role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=c6dad5af55404a81a0a649f9204e78ff) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:46.026819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:46.028254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d302c-a325-4668-871f-70a016b180a6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:46.034044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 229/455] 10.222.0.52 () {56 vars in 1376 bytes} [Thu Aug 4 20:41:45 2022] GET /identity/v3/role_assignments?group.id=b236edb55d4a41c3b3ce263ad9bd116c&scope.project.id=c6dad5af55404a81a0a649f9204e78ff&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:48.392898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0448d-1e26-44fa-a015-44e59af41faa None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.393454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0448d-1e26-44fa-a015-44e59af41faa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.393454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0448d-1e26-44fa-a015-44e59af41faa None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.395404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 227/456] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:48.401381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.401909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.401909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.429436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:48.430652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:48.530729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 230/457] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:48.555801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0448d-1e26-44fa-a015-44e59af41faa None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:48.573751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:48.684908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0448d-1e26-44fa-a015-44e59af41faa None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:48.724701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:48.725722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.726196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.726672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.745172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:48.746903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-8ab79f4c-d944-4979-9b91-cd660367e644 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:48.765795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:48.766683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:48.767506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:48.767506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:48.767506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:48.767506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:48.771438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 228/458] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:48.776707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:48.924269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f5c934-4943-4ba1-b7eb-dea77544d18c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:48.963080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:48.977097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:48.977749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.978037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.978316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.978433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81513bc2-211e-46b2-98ed-f6412c2b3996 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.978596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81513bc2-211e-46b2-98ed-f6412c2b3996 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.979026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:48.979332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:48.979434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81513bc2-211e-46b2-98ed-f6412c2b3996 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.980758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47c74-8def-47b6-992e-c679d5bc57cb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:48.981250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 229/459] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:41:48.993851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 231/460] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:48.996990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e76fc-bc41-491a-a5a9-b068fd2be3a0 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.997531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e76fc-bc41-491a-a5a9-b068fd2be3a0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.998021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e76fc-bc41-491a-a5a9-b068fd2be3a0 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:48.998969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf532b-8b93-4cc7-8edb-1176a4e50d48 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:48.999263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf532b-8b93-4cc7-8edb-1176a4e50d48 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:48.999543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf532b-8b93-4cc7-8edb-1176a4e50d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.000426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 230/461] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:49.001373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e76fc-bc41-491a-a5a9-b068fd2be3a0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.020964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4bdf532b-8b93-4cc7-8edb-1176a4e50d48 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:49.022332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4bdf532b-8b93-4cc7-8edb-1176a4e50d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:49.087573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.126156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 232/462] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:49.127176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e76fc-bc41-491a-a5a9-b068fd2be3a0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.127712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b70a-322a-4548-8069-2d16efb0642b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.128001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b70a-322a-4548-8069-2d16efb0642b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.128275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd5b70a-322a-4548-8069-2d16efb0642b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.152233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-7fd5b70a-322a-4548-8069-2d16efb0642b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:49.153447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7fd5b70a-322a-4548-8069-2d16efb0642b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:49.166365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5706a-666b-4339-a193-a482810856fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.167316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5706a-666b-4339-a193-a482810856fe None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.167847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5706a-666b-4339-a193-a482810856fe None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.168313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5706a-666b-4339-a193-a482810856fe None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.169305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5706a-666b-4339-a193-a482810856fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.171159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e5706a-666b-4339-a193-a482810856fe None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.197617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 231/463] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:49 2022] POST /identity/v3/users => generated 246 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:49.198448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5706a-666b-4339-a193-a482810856fe None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.247370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 233/464] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:49.249088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e437-1975-4119-a657-fa368c90a0f2 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.249612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e437-1975-4119-a657-fa368c90a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.250077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5e437-1975-4119-a657-fa368c90a0f2 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.256516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 234/465] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:49.258788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.259292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.259764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.293017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:49.294650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:49.346039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.360989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e5706a-666b-4339-a193-a482810856fe None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.392936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 235/466] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:49.394919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.397371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.420254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.420982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.421261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.421534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.434887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.436518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:49.454099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:49.454986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:49.455793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:49.455793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:49.455793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.455793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:49.456454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 232/467] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:49.456956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.549537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecaddb-a618-4cbd-9983-41c9fd66e5f2 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.574366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231c5b-e959-4bbf-b601-8ff201f798c5 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.602108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.602996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.603566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.604003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.612255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.613224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.614149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.615068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.616398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.618170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.626996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.628676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:49.636504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:49.637357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:49.638172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:49.638172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:49.638172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.638172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:49.638172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 236/468] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:49.638818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:49.650449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:49.651381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:49.652203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:49.652203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:49.652203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:49.652203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:49.652203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 233/469] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:49.653355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.680797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.684570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.755545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b0932-36eb-4a36-93d6-18f88b6b3bb3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.767192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fb47c4-fd13-46fa-a3a8-d955196ea2c3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.791928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.792914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.793438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.793892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.794754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:49.795230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.797051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.805864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.806747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.806857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.807203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.807997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:49.808387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.809857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.812867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 237/470] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:49.813930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.830432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 234/471] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:49.832915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:49.923917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.939006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd70e0-3c22-43b7-bc72-555014390aff None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.944967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:49.964186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2caa0d-153f-48ce-8178-251ffbca2c1b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:49.980310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:49.981057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:49.981492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:49.981777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:49.982645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:49.982954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:49.984439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:49.999397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 238/472] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:50.000412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:50.007543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.008171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.008485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.008768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.009466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:50.011452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:50.036760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 235/473] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:49 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:50.037669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:50.139363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd2457a-0254-4f7b-90f0-fe6d624ead8a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:50.183126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c74050-de7c-4199-b728-ad609ae71546 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:50.189918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.191435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.192057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.192057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.193495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:50.195387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9cebd-5d73-4064-8f49-e3bd4543eb40 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:50.220916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 239/474] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:49 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:50.222209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b517dc-8a4c-429a-a136-539fe84d291e None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.222504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b517dc-8a4c-429a-a136-539fe84d291e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.222774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b517dc-8a4c-429a-a136-539fe84d291e None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.225643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 240/475] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:50.230724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.231415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.231703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.231981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.232807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:50.234304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:50.234759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.235247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.235721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.244977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:50.258080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:50.262487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:50.264393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:50.264984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 236/476] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:50 2022] POST /identity/v3/users => generated 249 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:50.359359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 241/477] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:50.384732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:50.507459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2a9a-f56a-4cf1-995d-c756d4b3b3ca None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:50.547194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.547835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.548124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.548461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.549174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:50.550605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:50.563003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.sql.core [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 20:41:50.563003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 20:41:50.563003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [parameters: {'id': '6eda50c6692f4454a1cfb6bf11f72823', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 20:41:50.563003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86872) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result.read() Aug 04 20:41:50.580502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:41:50.581397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application yield resource Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise self.value Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application session.commit() Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 20:41:50.582378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.session.flush() Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application compat.raise_( Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 20:41:50.583672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:41:50.584638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result.read() Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:41:50.585502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [parameters: {'id': '6eda50c6692f4454a1cfb6bf11f72823', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:50.586683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 20:41:50.587782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:50.589050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 237/478] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Aug 4 20:41:50 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:50.593481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:50.704133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74616193-5e01-41b4-9db6-f2aa991145e8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:50.742876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.743769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.744196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.744643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.746747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:50.756913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f00b9ef9-e03d-4ec0-8ed4-0348bb773736 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:50.763545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:50.764917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:50.765687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 242/479] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:41:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:50.769420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:50.869669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:50.923466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d0bf3-e636-448f-b9de-17c3c13d2bcc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:50.965949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:50.966836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:50.967178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:50.967456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:50.968170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:50.968505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:50.969883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8003b392-1cee-4a65-a336-69d9dfb4ecdf None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:50.979609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 238/480] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:51.773860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6616-51e0-47a6-99a6-2f59cd46b032 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:51.774362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6616-51e0-47a6-99a6-2f59cd46b032 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:51.774471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841d6616-51e0-47a6-99a6-2f59cd46b032 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:51.776150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 243/481] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:51.793095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:51.793602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:51.793673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:51.825978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:51.827195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:51.970516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 239/482] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:52.013708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d6616-51e0-47a6-99a6-2f59cd46b032 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:52.234061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841d6616-51e0-47a6-99a6-2f59cd46b032 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:52.318230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c877e-b676-4e81-9341-aef67218afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:52.318884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c877e-b676-4e81-9341-aef67218afca None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:52.319164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c877e-b676-4e81-9341-aef67218afca None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:52.319437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c877e-b676-4e81-9341-aef67218afca None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:52.349274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c877e-b676-4e81-9341-aef67218afca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-945c877e-b676-4e81-9341-aef67218afca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:52.354725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:52.355576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:52.356594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 244/483] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:52.367322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:52.460617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:52.554382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86fbf4d-e807-4955-8e59-2738f3f430d7 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:52.622903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:52.623819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:52.624297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:52.624788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:52.625679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:52.626157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:52.627716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d90d0d-b811-4307-b1fc-17b453de9d8d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:52.639842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 240/484] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:52.645745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c877e-b676-4e81-9341-aef67218afca None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:52.659820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a0f8-03e5-4f98-badd-a6b252d9542a None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:52.659981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a0f8-03e5-4f98-badd-a6b252d9542a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:52.660260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4a0f8-03e5-4f98-badd-a6b252d9542a None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:52.661913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 241/485] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:52.666990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:52.667278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:52.667558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:52.686794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:52.688120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:52.750354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:52.783315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 242/486] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:52.821746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c877e-b676-4e81-9341-aef67218afca None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:52.832080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:52.860861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:52.861512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:52.861799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:52.862081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:52.874523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:52.876016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:52.892366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:52.893261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:52.894087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:52.894087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:52.894087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:52.894087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:52.894087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 245/487] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:52.900974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:52.965481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.031677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26478e0-1147-4b78-866c-a8b5ccbd7179 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.034184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cb6bc-2da7-4cf8-a4ff-fd9be262637c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.041519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.068139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.068814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.069093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.069368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.072306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.073202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.073577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.073923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.074689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:53.075063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.076597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b150a-005b-46f2-a762-8ace4c7f36e5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.080321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c60cf2f5-363f-4426-9b1d-201b3c3aada1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:53.085994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:53.086870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:53.087654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 243/488] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:53.089015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b0ff-89ac-4407-bfd0-debf92df8c98 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.092325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b0ff-89ac-4407-bfd0-debf92df8c98 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.092325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5b0ff-89ac-4407-bfd0-debf92df8c98 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.092325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 244/489] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:53.093304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5b0ff-89ac-4407-bfd0-debf92df8c98 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.094876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 246/490] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:53.101166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.101450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.101729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.121309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:53.122602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:53.204950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5b0ff-89ac-4407-bfd0-debf92df8c98 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.208499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 247/491] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:41:53.209401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.218893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.247769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a36138-dc39-457c-917e-b52918164765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.248512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a36138-dc39-457c-917e-b52918164765 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.249031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a36138-dc39-457c-917e-b52918164765 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.249139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a36138-dc39-457c-917e-b52918164765 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.249783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a36138-dc39-457c-917e-b52918164765 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:53.250087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a36138-dc39-457c-917e-b52918164765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.251462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a36138-dc39-457c-917e-b52918164765 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.261254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 245/492] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:53.262532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032a0d-c74a-41db-8a70-65b669bbf165 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.262821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032a0d-c74a-41db-8a70-65b669bbf165 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.263099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32032a0d-c74a-41db-8a70-65b669bbf165 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.268389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 246/493] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:53.268389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032a0d-c74a-41db-8a70-65b669bbf165 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.331642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.347371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb3cd35-3527-4ca8-8cbf-3a574c4d8ff9 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.387501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.388150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.388453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.388721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.399603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.401121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.406740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32032a0d-c74a-41db-8a70-65b669bbf165 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:41:53.418580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:53.419573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:53.420531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 248/494] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:53.421084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.453366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.453928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.454267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.454413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.467507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-060cb6ae-a282-4b33-9c14-94e2b299c726 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:53.473987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:53.474883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:53.475695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 247/495] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:53.477034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.477034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.477034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.497321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:53.498503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:53.527095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.546375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60dc282-4e9e-4275-936f-6fb5777ebe66 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.584444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 248/496] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:53.584444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.584669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.584669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.584669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.584669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.595331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.596832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:53.612263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:53.613232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:53.614634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:53.614634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:53.614634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:53.614634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:53.614634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 249/497] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:53.615479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.630057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.697726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35add9aa-41c3-410f-8189-a114a6d26b57 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.730482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fda0dbd-4ec2-4757-a7ac-76a2b4ae68a5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.732943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.733685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.733882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.734124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.734929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:53.735241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.736709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.764475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.765082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.765360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.765629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.766329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:53.766630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.767984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.775542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 249/498] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:53.776220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.783479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 250/499] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:53.784138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.806154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.853642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:53.885419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093658-fb06-4518-8eeb-5faf738dc155 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.891052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4f494-62bc-43bc-9fa0-663e952618e3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:53.921772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.922380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.922672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.922936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.923618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:53.923913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:53.925161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:53.925323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5bd98-0a16-487d-aeb0-0c577eda523c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:53.925812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.926677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.927269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.937337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:53.937995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 251/500] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:53.939258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570845fb-40c5-4330-9bda-253361a48675 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:53.939547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570845fb-40c5-4330-9bda-253361a48675 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:53.939821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570845fb-40c5-4330-9bda-253361a48675 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:53.941197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 252/501] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:53.944458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570845fb-40c5-4330-9bda-253361a48675 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:53.944584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:53.945420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 250/502] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:53.946226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.022955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.046685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31b4da-cab3-432f-a34a-40b1758ad039 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.047076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570845fb-40c5-4330-9bda-253361a48675 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.076414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.081846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.082586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.082900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.083016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.083286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.083508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.083787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.084063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.093567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.095040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:54.121706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.122585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.123741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:54.123741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:54.123741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.123741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:54.123741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 251/503] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.124372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.175565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.177509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d666896-50bc-4af3-b9a8-998ff204319f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.234877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878391c-f777-439b-a600-9da8f323ed59 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.261206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.269143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.270364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.270880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.271187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.277005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 253/504] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:54.278560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.278853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.279161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.282566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.284023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.297232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:41:54.297425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.298392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:41:54.304213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:54.304865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:54.304967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 252/505] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.304967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.327457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.372910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 254/506] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:54.373736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.409796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b55330-c879-4745-a441-008b8ca7ba00 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.444205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.444863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.445149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.445431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.446140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:54.446454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.447839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.455736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 253/507] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:54.456544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.492953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac990d53-2d2f-4159-b7e4-f12d9807b208 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.513874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.531793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.532450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.532729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.533006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.533690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:54.533991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.536524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.547654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 255/508] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:54.548487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.572116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.575039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43d5e8-8c28-4e2e-af64-fdefd6967fe9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.608005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.608677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.608963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.609246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.620488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.621962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:54.636601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.637427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.638228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:54.638228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:54.638228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.638228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:54.638228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 254/509] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.638518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.654587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7710d-2c3d-4840-b412-bb07d51cac89 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.689309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.689925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.690213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.690492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.691284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:54.691776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.693253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.731432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 256/510] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:54.732261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.738046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984143d-4fdd-43be-94ef-e30e567a1352 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.739654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.774941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.775568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.775848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.776201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.787273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.788784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:54.803677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.804558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.805393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:54.805393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:54.805393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.805393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:54.805393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 255/511] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.805653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.810443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:54.834015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b13e3-500e-4dfc-9b48-ede28f0c7231 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.868823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.869465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.869746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.870020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.880635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.882129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:54.897416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.898343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.899201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:54.899201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:54.899201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:54.899201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:54.899201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 257/512] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.899472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.912566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5a8dc-070c-47ce-a2c3-f9feb704fb8e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:54.947040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:54.947667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:54.947944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:54.948215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] PATH_INFO: `/v3/users/glance` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:54.955445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:54.956990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:41:54.968216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:54.969199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:54.970068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 256/513] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:54.970637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:54.989936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.008138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b924f-8d4f-4b3d-b09e-4821c0b61f0a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.037944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.044316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.044944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.045224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.045498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.046181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:55.046479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.047843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.059919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 258/514] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:55.060774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.080484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27495b4d-2b36-4592-ba23-8e3ac0135821 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.115350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.115969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.116253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.116562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.168961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0dbea5-002d-4496-ae6a-ab251a1942e2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.183769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.185250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.200481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.204986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.205704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.206055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.206397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.208138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:55.208478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.209859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.221368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 259/515] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:55.222185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.273045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.289680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 257/516] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:55.290638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.325246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec201f5d-52fb-4ce7-9e55-5a82edf5ebc5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.359133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.359745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.360024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.360295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.360996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:55.361297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.362686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.374260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 260/517] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:55.375071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.392768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a30a66-adc0-438b-87cf-08f4fe46a926 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.410028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.430886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.431423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.431700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.431969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.432786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:55.433090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.434509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.452151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 258/518] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:55.453073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.479579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637d79-0c53-44bf-8df8-8f74d1448632 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.490568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.516477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.517274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.517617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.517982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.525884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.527194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:41:55.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:55.540896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 261/519] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:55.546678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.571751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943aa85-517a-4d29-8fee-1f91b2dd4781 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.606153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.606858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.607215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.607503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.608212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.609706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.632876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 259/520] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:41:55 2022] POST /identity/v3/users => generated 247 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:41:55.634019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.656015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e5eb03-7579-4ad2-b9da-bcd5031a4799 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.667850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.698795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.699448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.699798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.700160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.711522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.713146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:55.729347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:55.730260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:55.731074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:55.731074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:55.731074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:55.731074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:55.735304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 262/521] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:55.736096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.758287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff60ce-4ef0-478d-9c83-ae6561d7303c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.762252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.799611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.800369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.800662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.801458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.811645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.812963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:41:55.825344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:55.826453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:55.827528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 260/522] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:55.828157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:55.861605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c80a3-308b-44eb-99d2-c15d535d5981 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.898128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.898799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.899014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.899280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.926800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:55.941308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b6328-8af0-42c6-bafb-484f7f2f785d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:55.970012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.971371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.978654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:55.979398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:55.979691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:55.979970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:55.980758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:55.981190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:55.982783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:55.994915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.009509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 261/523] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.009509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.083734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 263/524] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:56.083734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.116592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af96db-ee9f-4b1a-a298-e763f6b948ce None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.154585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.155320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.155991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.156724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:56.157051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.158396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.171545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 262/525] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.172399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.185243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.198869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe78a86-88f9-416a-8d99-cb7616bd53d7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.223330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.236645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.237256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.237541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.237802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.238992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:56.239304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.240792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.259928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 264/526] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.265338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.283960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952484f2-5dfd-4f17-a3cc-30f1062aba6b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.321762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.322376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.322664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.322939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.323724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:56.324034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.325544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.331179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 263/527] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.332128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.383940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8fe060-03e5-4195-8415-8ef444e87ba7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.426817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.427842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.428125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.428459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.440150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.441323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.446711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:56.460318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:56.461281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:56.462099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:56.462099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:56.462099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:56.462099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:56.462099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 265/528] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:56.462391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.501033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02cff32-9268-4c96-887c-0cb6034d3499 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.513344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.545345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.546222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.546650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.547172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.555159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.556844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:41:56.569676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:56.570711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:56.571710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:56.575543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 264/529] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:56.576396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.589376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf19e2-8d7a-42a9-8b39-81089dbdeade None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.636371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.636371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.636371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.636371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.700847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11736678-0ccc-411b-b47c-1821e8cc8c09 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.720212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.722007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.733676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.741529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.742348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.742718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.743106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.743897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:56.744526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.746088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.759727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:56.761087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 265/530] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.762256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.849002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 266/531] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:41:56.849912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.887486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ec5c3-53a3-4bed-b3c4-a077d7237818 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:56.935004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:56.935634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:56.935887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:56.936156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:56.936927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:56.937314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:56.938650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:56.955859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 266/532] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:56.956758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:56.991842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5a9f3-76c7-4b52-b91f-673625d64302 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.013027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:57.040629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:57.050105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.051673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.052669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.053132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.063560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.065886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.072366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c5d12-30ae-49aa-9a1f-74c80641d722 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:41:57.081336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:57.082245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:57.083075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:57.085720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 267/533] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:57.086674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.111264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.112051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.112460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.112896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.124587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.126184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:57.142443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:57.143518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:57.144568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:57.144568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:57.144568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:57.144568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:57.150835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 267/534] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:57.151836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.209889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86fdd37-b322-4c85-9e74-c909c67d5a08 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.247276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.248039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.248483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.248851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.250159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=49153b6e04e24c8d93d323d0a4757684) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:57.250159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.251767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.264705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 268/535] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/role_assignments?user.id=49153b6e04e24c8d93d323d0a4757684&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:57.265679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.292008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4def3-2960-43a8-9a45-d10f5b82412e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.295275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:57.339111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.340139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.340724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.341157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.342086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:57.342563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.344539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.362511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 268/536] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:57.368114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.397582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:57.521209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362dca41-bfd1-4e40-b4b4-0c15dcfbf53e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.548618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b034016-8f4d-4bff-a97d-2b119a68d03e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.591852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.592510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.592802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.593083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.594236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=f260187ace2043849cb3a4ab780899a9) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:57.594551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.595997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.603331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 269/537] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/role_assignments?user.id=f260187ace2043849cb3a4ab780899a9&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:57.608086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.608756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.609041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.609351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.610035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:57.610343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.611758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7746c2-11ab-402c-a048-b7a72580ab58 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.624814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 269/538] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:57.633517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.682226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:57.795421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc7850-1b1b-4e7d-a048-9cd2421819ce None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:57.836659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:57.837655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.838160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.838736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.843465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a082-891b-4ce4-8810-22c5792c3a59 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.843760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a082-891b-4ce4-8810-22c5792c3a59 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.844041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422a082-891b-4ce4-8810-22c5792c3a59 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.846157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 270/539] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:57.852368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:57.852763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:57.853078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:57.855086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:57.856801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:41:57.871191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:57.872067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:41:57.872934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:57.875249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 270/540] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:57.880401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:57.887295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:57.889639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:58.002327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 271/541] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:58.036968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:58.038624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.052701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b213a-9bbb-459f-8473-424d01930b9d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:58.089582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:58.090207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.090487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.090760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.091646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:58.091908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:58.093709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:58.107065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 271/542] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:58.115466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.166654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:58.237745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33109ebe-7893-4876-8c1f-b9fa6943dfd0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:58.274485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:58.275101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.275381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.275651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.276468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=9ef5ca1eeba741259394f0ab37dd3751) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:58.276777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:58.278217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:58.284552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 272/543] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/role_assignments?user.id=9ef5ca1eeba741259394f0ab37dd3751&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:58.292544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c294eb-421c-47bc-8ca8-cc7c5582e17d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:58.355411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:58.356041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.356325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.356632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.368244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-bc26c353-d8f4-4182-ac3a-1bf5e20b164b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:58.373333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:58.374249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:58.375610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 272/544] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:41:58.382140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.417487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:58.525859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7b99-ec42-4985-be13-9fbfc4a74014 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:58.558866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:58.559481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.559759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.560036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.560779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:58.561082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:58.562590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:58.567202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd43b7-dc3a-4b33-93d8-bc729c9c32aa None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.567488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd43b7-dc3a-4b33-93d8-bc729c9c32aa None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.567757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd43b7-dc3a-4b33-93d8-bc729c9c32aa None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.569264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 273/545] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:58.575695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 273/546] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:58.577815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.578161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.578438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.582159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.617446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:58.618545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:58.705933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:58.744267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 274/547] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:58.754749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98508c6d-e9b8-46e5-b2d5-a19bd42e42da None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:58.781758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.814425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:58.815132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:58.815414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:58.815688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:58.838426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:58.841931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:58.843509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:58.868331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:58.869217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:58.870028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:58.870028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:58.870028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:58.870028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:58.870239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 274/548] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:58.876189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:58.982588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f8f38-2c97-46eb-9faa-f5fd95fbd4e2 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.057668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d0703-e016-48fb-bfb5-674a1dd6a2b9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.069964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.070680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.070968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.071244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.100922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:41:59.101512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.102494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.104691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.105183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.106116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:59.108589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.110294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:41:59.117277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:59.118791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:41:59.119761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:59.126504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 275/549] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:59.132298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.143023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 275/550] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:59.150615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.265914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb29178e-4fba-4ddd-9996-72c1f14286ca None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.276934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.300857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.301476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.301846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.302379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.303092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:59.303397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.304804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.313369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 276/551] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:59.319495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.373157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014caec9-55b3-42c6-aebe-397364cc1708 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.433900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.434542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.434820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.435088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.443868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.445606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:41:59.459659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:59.460721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:41:59.461687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:41:59.463659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 276/552] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:59.482177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.517165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa667d-af45-413f-84a9-6dc310fedc36 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.568764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.600629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.601289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.601572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.601848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.628266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.629824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.639260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669e913-3081-4c01-b236-6ef4fcddde71 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:41:59.663441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:41:59.664397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:41:59.665951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:41:59.665951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:41:59.665951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:41:59.665951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:41:59.665951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 277/553] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:41:59.670640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.688741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.692308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.693041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.693326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.693680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.694488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:59.694797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.696254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abc0d6-9368-4c2b-8c62-840df7443583 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.720593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 277/554] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:41:59.721943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80455-906d-4a07-90be-2d48c1930cbe None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.722236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80455-906d-4a07-90be-2d48c1930cbe None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.722547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe80455-906d-4a07-90be-2d48c1930cbe None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.724386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 278/555] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:41:59.725745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80455-906d-4a07-90be-2d48c1930cbe None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:41:59.791018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefe5d0-36f6-4316-a1c7-6ce2a5575fc1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.837051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.837284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe80455-906d-4a07-90be-2d48c1930cbe None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:41:59.837732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.838017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.838291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.839182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:41:59.839605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.840281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.841721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d794a-6d58-4a85-9719-880a4a8cc968 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.862330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 278/556] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:41:59.862330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.862330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.862330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.882179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:41:59.882502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:41:59.883363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:41:59.883488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:41:59.883954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:41:59.884467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:41:59.961023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:41:59.962855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:41:59.965525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:41:59.976370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 279/557] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:41:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:41:59.976370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.096037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 279/558] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 20:42:00.096592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274b70f-d170-4600-8ef8-29c7861cc9db None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.099212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.103917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.135297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.135863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.136170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.136462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.143917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.146183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-9875ed3c-5ed2-43f5-b18e-a9a54b4f9db4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:00.158899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.161337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:00.162454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:00.164870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 280/559] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:41:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.166253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d27c-ccbe-4175-87ee-633092f474bd None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.166545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d27c-ccbe-4175-87ee-633092f474bd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.166826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0d27c-ccbe-4175-87ee-633092f474bd None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.168138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 281/560] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:00.168859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d27c-ccbe-4175-87ee-633092f474bd None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.225758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d6e6-cbd3-426c-9540-c8bca9519369 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.240704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.268268 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.270040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.270342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.270813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.285635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:00.294526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.295550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:00.296434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 280/561] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.297746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.303591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0d27c-ccbe-4175-87ee-633092f474bd None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.339748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.340405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.340687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.340957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.352275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.354199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-3049864d-fb0e-4275-9874-25d36abde2ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:00.371171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.372099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 282/562] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.374065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.384615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.393757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:00.394991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:00.426513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d63c137-e5f9-488b-b105-05ffd521b326 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.465727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.466345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.466749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.467075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.468235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:00.468400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.469812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.477178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 283/563] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:00.478025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.501673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 281/564] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:00.502863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.539503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.600160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b264fbc-c02d-481b-b955-f960a391ccca None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.611553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621f209-02c6-49b7-bc1d-f439d81c10b8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.634013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.634623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.634909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.635021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.635177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.636357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:00.636735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.638420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.647278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 284/565] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:00.648081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.648220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.648725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.649007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.649279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.649985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:00.650283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.651713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.663775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 282/566] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:00.665372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.747553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9616ba-2402-4f47-ab90-11af6e906635 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.749849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.764327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0f03c-7922-4d02-a56d-3c2630d54dfd None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.783004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.783646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.783928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.784198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.794325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-6023516c-e337-42b6-9e98-8de889078ef3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:00.799556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.800620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:00.801511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 285/567] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.802473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a00b-1a62-4d16-b9d2-44e47683767a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.802571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a00b-1a62-4d16-b9d2-44e47683767a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.803125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421a00b-1a62-4d16-b9d2-44e47683767a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.804219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 286/568] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:00.808512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421a00b-1a62-4d16-b9d2-44e47683767a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.814518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.815175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.815458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.815734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.828329 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.829814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:00.845722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.846847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.847794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:00.847794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:00.847794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.847794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:00.847794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 283/569] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.848728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.885337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:00.929306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421a00b-1a62-4d16-b9d2-44e47683767a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.953039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcc03f-478e-44ae-94a3-173b48d3fd10 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:00.963452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.964087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.964388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.964667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.975640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:00.977357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:00.986633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:00.987273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:00.987556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:00.987832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:00.992253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:00.993139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:00.994008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:00.994008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:00.994008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:00.994008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:00.994008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 287/570] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:00.994292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:00.999162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.000660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b64b00a1-2392-459b-825e-63fda87ae65f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:01.012668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:01.013715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:01.014864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 284/571] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:01.015453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.015599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.015875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.021597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.034731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:01.036552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:01.103689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b18ef9-44d9-49c9-8cd5-9e0eaf418af6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.104212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 285/572] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:01.104688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.134621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.146908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.147610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.147960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.148311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.149045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.149348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.150720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.161965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 288/573] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.163131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.212627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48373d-a01d-4e14-9647-2a550ce9d197 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.243764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.251730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.252498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.252884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.253273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.254800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.255205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.256668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.266795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d1a57-9105-4de3-9674-a78449170ed3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.267842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 286/574] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.270300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.302837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.303441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.303714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.303977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.304679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.304978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.306331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.318421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 289/575] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.319213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.356435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.372629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f97473-a814-4647-82d3-e5f66f5a62c5 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.406195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.406807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.407090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.407364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.408108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.408447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.409893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.419938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593910-133e-4ff3-803a-3e7fc5e6cd2c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.422412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 287/576] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.423193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.455614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.456258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.456569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.456960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.464082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.467794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:01.473176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:01.474132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:01.475461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:01.476556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 290/577] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:01.476771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.542112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409b732-e6cd-4694-9a36-a4fae4f2e0c3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.576003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.576674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.576953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.577224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.579151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b91fde-46ab-4eaf-b110-de11b2a4511e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.587828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.589300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.591772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:01.604574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:01.605478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:01.606338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:01.606338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:01.606338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:01.606338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:01.606338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 288/578] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:01.606900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.618628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.619264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.619547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.619817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.626563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.628009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:01.640415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:01.641482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:01.642758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 291/579] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:01.647577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.704877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.705837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f5c10-84d6-4135-8483-5f4192077672 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.739578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.740145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.740471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.740756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.746253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0adc93-b1f9-4457-ada5-7c6d026ed764 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.748228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.749695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:01.761871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:01.763080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:01.764068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 289/580] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:01.765743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.781720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.782384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.782670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.782945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.783731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=352be9523cf1454a86c16640a87ece9c) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.784038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.785869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.791462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 292/581] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/role_assignments?user.id=352be9523cf1454a86c16640a87ece9c&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.792241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.830093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.886982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35c4a2-f68f-417e-91db-19409330e69d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.903405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67551828-60d1-4519-98ed-5984d4c69825 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:01.927430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.928067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:01.929586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.930126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.930388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.931138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.931463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.932924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.941994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:01.942620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:01.947253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:01.947253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:01.947474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 290/582] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.947474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:01.947642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:01.947642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:01.947642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:01.968214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 293/583] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:01.968214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.062255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b42aeb-f9ff-4249-9ee1-150128ec7597 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.075958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941064c-95fc-4d3c-8188-ee8dfd00d62d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.080751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.100069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.100724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.101003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.101275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.102082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:02.102394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.103835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.113656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.114383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.114659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.114935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.115730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:02.116064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.121359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 291/584] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:02.121777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.121875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.138846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 294/585] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:42:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:02.139499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.186003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.234657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603abb44-37cc-497e-b893-8a61882bc0a4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.255491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1971fe14-8212-41d3-b1e9-ccd57e6cbf5c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.272817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.273690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.274118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.274537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.285349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.286967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.291537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.292177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.292490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.292767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] PATH_INFO: `/v3/users/glance` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.300085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.301604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:02.302355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:02.303201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:02.304056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:02.304056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:02.304056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:02.304056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:02.308064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 292/586] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:02.308901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:02.317218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:02.318141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:42:02.319076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:02.323211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 295/587] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:02.324034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.343834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.417012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f41dc-3624-451b-9a13-944814a4e66d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.432548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.436465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5386054-1195-4f10-a786-55f624e8d07b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.453699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.454373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.454653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.454924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/78496973911641848ac1ee3031c692e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.470863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=02a04385570944ecbcaf96003f5c5753, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.472302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.478427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.479272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.479555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.479879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.496377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 293/588] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:02 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:02.496377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.496661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.496661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:02.514333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:02.515295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:02.516253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:02.516253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:02.516253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:02.516253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:02.518366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 296/589] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:02.524574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.615607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.641227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62d1ef-654c-40c8-91e2-1740b733edef None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.657119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e85eb-5031-45f2-b7ed-9725665d3c83 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.682478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.683408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.683518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.683839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.684648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:02.684957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.686596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.699136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.699751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.700033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.700307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.700854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 294/590] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:02.701238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:02.701554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.701716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.703121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.723360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 297/591] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:02.725584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.745698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.822818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb84a3-0941-461d-b2c5-247c9cf842e8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.857941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661a912-0329-4f67-a2b9-066a0d86197b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:02.876380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.877157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.877447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.878101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.879475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:02.879922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.881526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.895389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:02.904254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 295/592] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:02.905607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:02.912103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:02.913168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:02.913603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:02.914050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:02.928635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:02.930344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:02.946405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:02.947590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:42:02.948625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:02.950696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 298/593] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:02.951733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.026495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.036539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c69fa3-208c-47ec-ae94-89ef35191201 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.062769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c748d7e2-5d84-4e74-9266-51378ea20ab8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.083686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.084354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.084659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.084932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.099532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.101059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.102657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.103468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.103897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.104387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.111847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.113724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:03.118182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:03.119214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:03.120271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:03.120271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:03.120271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:03.120271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:03.124100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 296/594] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:03.125118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:03.131498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:03.132603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:03.133597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:03.135572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 299/595] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:03.136517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.201194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.263078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215faaa4-45b0-4e4a-82c2-54d7d70422be None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.263395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d381893-e6a3-43f6-ad97-ba784ad459bb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.293918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.301630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.302475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.302760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.303029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.303722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:03.304020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.304423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.305205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.305432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.305800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.306232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.307357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:03.308141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.309754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.326867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 300/596] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:03.327792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.334892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 297/597] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:03.336001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.432778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545778b-7858-4a07-a2cc-13c3149bceb9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.448576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465ad7c-d0a2-41b3-ba89-fa68ffb2031a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.460225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.469467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.470541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.470976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.471280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.472454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:03.472777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.474771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.492166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.493087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.493609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.494032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.499213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 301/598] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:03.500123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.502403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.504845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:03.517393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:03.518468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:03.519533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:03.521582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 298/599] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:03.522487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.560935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.619083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a56be5-5490-4774-bea9-c81159542280 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.637989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad37ff-4590-4baf-a099-f8f4ad15c6e4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.659600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.660486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.660967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.661321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.678441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.680110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.690254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.691008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.691968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.692363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/49153b6e04e24c8d93d323d0a4757684/roles/78496973911641848ac1ee3031c692e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:03.703528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:03.704672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:03.705772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:03.705772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:03.705772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:03.705772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:03.708462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 302/600] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:03.709423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.718148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=49153b6e04e24c8d93d323d0a4757684, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.719841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.736849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 299/601] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:03 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/49153b6e04e24c8d93d323d0a4757684/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:03.738274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.772191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.851890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e21ef-d3ae-40d4-877b-0760dcf52827 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:03.854290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:03.895399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:03.896151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:03.896521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:03.896800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:03.897664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:03.898301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:03.899803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:03.916226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 303/602] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:03.922662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:03.977592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564aa5d-307a-432d-9779-d193fe40eca5 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.022475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.024057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.024583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.025087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.026521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:04.027441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.029454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94145da2-2ae4-483c-b27a-cc0a69f8d4d3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.046801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 300/603] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:04.052764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310b91a-b698-437e-82e5-d405210796c9 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.052764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310b91a-b698-437e-82e5-d405210796c9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.052764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310b91a-b698-437e-82e5-d405210796c9 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.052764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 301/604] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:04.057079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310b91a-b698-437e-82e5-d405210796c9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.059516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f9b99-6828-433b-abd7-8177e55f8ac4 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.099853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.100566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.100849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.101124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/f260187ace2043849cb3a4ab780899a9/roles/78496973911641848ac1ee3031c692e6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.118160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=f260187ace2043849cb3a4ab780899a9, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.119542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d20c92-e409-4819-9b94-c07f094a37bb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.139088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 304/605] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:03 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/f260187ace2043849cb3a4ab780899a9/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:42:04.141717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.142226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.142746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.187202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:04.190236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:04.197112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.206514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.243641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310b91a-b698-437e-82e5-d405210796c9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.312430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.313155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.313441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.313719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.327663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 305/606] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:04.328564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.330493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:04.343426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:04.344354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:04.345229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 302/607] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:04.351218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.369201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.469937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d8399-b4b1-45e5-81f1-4392f7ff0ff8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.488958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e9f71-3853-433e-abcb-a2a3f859344e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.507968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.508546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.508831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.509107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.509866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:04.510176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.511631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.528599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 306/608] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:04.530239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.530351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.531353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.531901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.532401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.533089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.544073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.548583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:04.556325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:04.557315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:04.558222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:04.562063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 303/609] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:04.568769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.638737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bc1d4-3303-4454-b357-76206ffc2ec8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.676367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.677014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.677601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.677880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/9ef5ca1eeba741259394f0ab37dd3751/roles/78496973911641848ac1ee3031c692e6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.680637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ad1ca-8b9a-4f06-98c2-4cdda8463b78 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.694331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=9ef5ca1eeba741259394f0ab37dd3751, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.696205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.711633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 307/610] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:04 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/9ef5ca1eeba741259394f0ab37dd3751/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:04.718445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.719480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.719971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.720484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.721438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:04.721986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.725007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3092b2b9-5e2d-406d-a831-cb71d2a584c0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.735514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 304/611] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:04.742655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.760791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538ec1-c94d-476a-bbd1-556b85b9761a None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.761156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538ec1-c94d-476a-bbd1-556b85b9761a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.761539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12538ec1-c94d-476a-bbd1-556b85b9761a None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.763257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 305/612] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:04.771870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.772315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.773212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.799243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:04.802816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:04.814871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.885347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:04.898039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 306/613] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:04.905436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d10afa5-145e-4e9e-887f-2daa42d13ee3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:04.923562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:04.955353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:04.956394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:04.956945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:04.957484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:04.973189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:04.974964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:04.994442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:04.995378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:04.996218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:04.996218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:04.996218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:04.996218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:04.998519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 308/614] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:05.005479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.078339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847c158-e9a7-4b3f-aadd-632ce405662f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.144647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.145638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.146111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.146578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.166463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b9b903-aea3-4188-8769-1b2432da8177 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.169007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:05.169806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:05.178504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:05.179419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:05.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:05.184410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 307/615] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:05.192542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.237028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.237938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.238986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.239408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.240250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:05.240718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.242263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.265685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 309/616] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:05.273346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.292734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:05.353872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f9469-1503-4465-ba32-ffdba83f5ab7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.416901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.417525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.417802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.418072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.418788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:05.419087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.420542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.436606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 308/617] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:05.440321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.466341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b78b4-27f5-4517-87e7-4c1b7240c95c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.532233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.532914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.533297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.533650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.541496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.543155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:05.556015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:05.557137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:05.558090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 310/618] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:05.563206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.582872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dd249-1799-4a42-bdec-5394a1e1fa57 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.602942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:05.615002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:05.656315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.656987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.657258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.657529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.678046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.679515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:05.708260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:05.709410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:05.710611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:05.710611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:05.710611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:05.710611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:05.713479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 309/619] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:05.717892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.752812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6d0de-256e-4d0b-874b-b7b8b6182b03 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.825865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.826836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.827261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.828296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.829278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:05.829745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.831332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.859620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 311/620] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:05.875740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:05.897403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092a340-82d1-4876-a76f-ada9d45c90fa None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:05.936596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:05.937225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:05.937509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:05.937786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:05.938485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:05.939905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:05.941377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:05.967212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 310/621] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:05.974378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.005267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.020738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.029348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc5ef-609d-4ae5-a541-901707b69463 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.065732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.066504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.067092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.067092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.077501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.078935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.083798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b8403-17dd-4efb-875a-09df050bf978 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:06.094772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:06.095686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:06.099087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:06.099087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:06.099087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:06.099087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:06.099087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 312/622] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:06.104205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.143977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.147424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.148448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.149151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] PATH_INFO: `/v3/users/nova` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.160906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.162789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b331d422-00ea-45ef-84c5-a6a68500390b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:06.180288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:06.181343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:06.182291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 311/623] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:06.182921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073af0c-7178-40dc-9d3c-238e37894a3e None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.182990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073af0c-7178-40dc-9d3c-238e37894a3e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.184750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073af0c-7178-40dc-9d3c-238e37894a3e None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.189357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 312/624] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:06.193676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073af0c-7178-40dc-9d3c-238e37894a3e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.251554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bbf5c-cadb-4191-9b81-6c90bc883370 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.287038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.297947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.299229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.299801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.300229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.301498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:06.302510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.304322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c5cf57-8cca-40a2-a668-3e2465b16041 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.306326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073af0c-7178-40dc-9d3c-238e37894a3e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.306844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.326362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 313/625] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:06.328292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.328868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.329427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.352450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:06.352865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.353678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.354125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.354310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:06.354722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.355676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:06.356143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.359016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.383762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 313/626] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:06.384869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.444923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 314/627] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:06.444923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.543957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a590e0-2c81-48a3-939c-4a0835731f3d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.574469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91a483-8dea-4548-847d-b966ab3e60bf None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.589835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.590636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.590927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.591205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.600786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.602398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.602754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:06.616062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:06.617167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:06.618172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:06.624543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.624543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.624543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.624543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.625011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 314/628] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:06.626087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.631181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.721885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.723897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d27618-3ad1-4e52-b742-edd446e966ca None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.757089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21774a3-ecae-4ae3-8435-7b450d8b0eb8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.805479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.806194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.806482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.806756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.819223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:06.825125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:06.826086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:06.826922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 315/629] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:06.829160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 315/630] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713bd4-8b9c-49b0-97b0-2a96de242d56 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713bd4-8b9c-49b0-97b0-2a96de242d56 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713bd4-8b9c-49b0-97b0-2a96de242d56 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 316/631] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:06.844505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713bd4-8b9c-49b0-97b0-2a96de242d56 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:06.872310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.908315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:06.950786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7e7bd9-0f9e-4e91-a006-f9977f0d31cf None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.955559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713bd4-8b9c-49b0-97b0-2a96de242d56 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:06.988974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.989363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.989706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.989989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.990758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:06.991057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.992917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:06.994634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:06.995266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:06.995546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:06.995827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:06.996550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:06.996846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:06.998313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac05dab-cac5-4cb9-87bf-745505e3dfad None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.012172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 316/632] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:07.012172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.013077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 317/633] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:07.014474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.014938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.015391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.037053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:07.039011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:07.119015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 318/634] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:07.119873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.123928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafaac7-090d-42e9-a6f8-02da0de5ef04 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.138517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.155105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.175003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.175845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.176135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.176453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.187549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.189063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:07.204056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:07.205283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:07.206149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:07.206149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:07.206149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:07.206149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:07.206149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 317/635] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:07.206441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.242274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776dc48-6a0f-4309-9d05-c2c942410041 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.280459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.281193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.281707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.282093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.294750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.296192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.314153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55827999-9aab-4a9f-ae39-28b5360a89ce None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-9815fa37-f502-4af0-a6c5-52efcf5f785e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:07.320458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:07.321539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 319/636] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0d13-bbba-4960-841a-5a6fec863ba5 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0d13-bbba-4960-841a-5a6fec863ba5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db0d13-bbba-4960-841a-5a6fec863ba5 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 320/637] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:07.330364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0d13-bbba-4960-841a-5a6fec863ba5 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.351279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.352049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.352366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.352645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/352be9523cf1454a86c16640a87ece9c/roles/78496973911641848ac1ee3031c692e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.368925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=352be9523cf1454a86c16640a87ece9c, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.370400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.387955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 318/638] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:07 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/352be9523cf1454a86c16640a87ece9c/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:07.389159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.423900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.440746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.479638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db0d13-bbba-4960-841a-5a6fec863ba5 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.528712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.529390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.529677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.529952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.539030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cd76a7-bcb8-4a9e-a7ea-8332242fabcf None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.543261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:07.548282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:07.549326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:07.550197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 321/639] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:07.550942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.591929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.593091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.593661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.594206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.595506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:07.596023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.598699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8a467-0f10-44e3-b26c-421582d212d1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.613578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 319/640] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:07.615138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.615620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.616096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.635343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:07.636706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:07.678875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54bb1f-fb6d-4e61-a101-0c7a37a5737c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.708939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.714808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 320/641] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:07.715374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.731558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.733150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.734018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.734550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.735284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:07.736864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:07.737634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.740351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.765340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 322/642] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:07.765340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.842078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576880c-393e-43af-a3af-93882a9a8e74 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.879972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.880632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.880920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.881196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.881916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:07.882219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.883646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.886501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1447ba5-5b96-4083-b656-3fc9ab3690b9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:07.898019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 321/643] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:07.898797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.925743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:07.926365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:07.926647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:07.926928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:07.933987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:07.935440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:07.946824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:07.947989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 323/644] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:07.949255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:07.999937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.009028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.026957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f16574-7c52-44f4-b5a2-4a52cf0e8db2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.063675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5faaac3-ab07-433c-8a33-de2d60dd25fc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.066753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.067403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.067684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.067956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.079063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:08.084519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:08.085639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:08.086862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 322/645] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:08.087652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.106737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.107404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.107682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.107952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.116232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.117719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:08.130600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:08.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 324/646] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:08.132592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.207651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ad144-b59b-4c99-b466-da3fe8c12ccb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.244195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435e8e7-d375-4631-a3eb-69bf6a548af0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.244767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.245492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.245841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.246134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.254181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.254580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.262208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.263751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:08.280434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:08.281319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:08.282172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:08.282172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:08.282172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:08.282172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:08.282396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.282396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.282871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 323/647] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:08.282871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.287505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.287918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.288749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:08.289068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.290581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.304067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 325/648] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:08.304929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.408976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08efd20d-3360-4f5f-be9c-df57cb27e635 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.416174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f9ea3-2e44-46e8-8c65-800cc9e0e6d9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.445122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69688774-fa89-435f-b321-7b293af62063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.445740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688774-fa89-435f-b321-7b293af62063 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.446021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688774-fa89-435f-b321-7b293af62063 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.446549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69688774-fa89-435f-b321-7b293af62063 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.446985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69688774-fa89-435f-b321-7b293af62063 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:08.447291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69688774-fa89-435f-b321-7b293af62063 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.448682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69688774-fa89-435f-b321-7b293af62063 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.454545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.455168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.455445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.455719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.456534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:08.457934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.462601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 324/649] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:08.462601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69688774-fa89-435f-b321-7b293af62063 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.462803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.481056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 326/650] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:08.486408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.525988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.541238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.596735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69688774-fa89-435f-b321-7b293af62063 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.617731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a59f63a-dc79-401f-9634-ab82c3632ebd None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.631281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.631902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.632178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.632471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.633153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:08.633449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.634826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.646948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 325/651] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:08.647735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.660518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.661440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.661967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.662424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.663359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:08.663818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.665667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.680567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 327/652] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:08.681576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.771631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e201a-bc4d-45b1-8e05-313963434039 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.788479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.797838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:08.812496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720ecea-61ab-483c-8c4e-4f7a3974d3cc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:08.815688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.817952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.817952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.817952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.830436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.832915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:08.850034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:08.851138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:08.852160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:08.852160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:08.852160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:08.852160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:08.855919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 326/653] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:08.857572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:08.875343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:08.875552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:08.876280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:08.876507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:08.980526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:08.982341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.004087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5264ba3a-0646-4a32-9cb9-0b639c6911ef None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.051185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.051845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.052412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.052696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] PATH_INFO: `/v3/users/glance` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.064439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.067526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:09.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:09.089709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 327/654] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:09.091057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.099775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.112228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 328/655] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:09.117941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.134575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.228535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d2847-5f2f-46e8-af74-65c87d69f83d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.246305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb22af-d6f5-4b7c-a1bc-6129cd8f64fa None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.267216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.267911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.268459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.268750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.270509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:09.270836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.272310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.290900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.291874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.292442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.292934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.293862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:09.294358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.295996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.300415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 329/656] 10.222.0.52 () {56 vars in 1137 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:09.301318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.317737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 328/657] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:09.318946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.375463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.411002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.430365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7279c0e-a1a4-4142-999b-f4566cb62760 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.434110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f8190-f584-42c1-9bc1-3bc4b017b681 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.467756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.469006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.469666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.470033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.470399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.470666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.470943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.471214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.482211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.483749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:09.500828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:09.501886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 330/658] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:09.502831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.562230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.563710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961c724f-dc14-4d77-a78f-823728c93a7e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.613792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.618060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47695b32-5902-4c0b-89c2-98af486eac0e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.649985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.654336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.654969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.655246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.655516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.663713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.665166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.669842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 329/659] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:09.670815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8885d53-1c22-4883-8e28-dd225cde44e4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.671105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8885d53-1c22-4883-8e28-dd225cde44e4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.671384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8885d53-1c22-4883-8e28-dd225cde44e4 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.672831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 330/660] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:09.678904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8885d53-1c22-4883-8e28-dd225cde44e4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:09.681885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:09.682792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:09.683709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 331/661] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:09.685017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.789942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c4ba6f-e899-4398-98b1-60db95e3143e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.790730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8885d53-1c22-4883-8e28-dd225cde44e4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:09.825159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.825902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.826255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.826526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.826699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:09.827270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.827554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.827830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.828542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:09.828846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.830213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ab87b-4694-4351-8907-89f061815ffd None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.837301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:09.838699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:09.846770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 331/662] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:09.846770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:09.846770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:09.846770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:09.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:09.857605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:09.858478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:09.858478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:09.858478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:09.858478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:09.858478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 332/663] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:09.858751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.868587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:09.869787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:09.874809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.918500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:09.947463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 332/664] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:09.948657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:09.986332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f0e8-1506-40f3-8914-ca5f7f29045c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.025478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.026091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.026373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.026645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.027438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:10.027742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.029200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.054671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 333/665] 10.222.0.52 () {56 vars in 1139 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:10.054671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.063384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5aa98a-d4d5-4ed8-881f-9b8ddd831ec4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.102784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.103425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.103707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.103979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.110962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.112438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:10.123872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:10.124826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:10.125739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 333/666] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:10.126212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.138010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.164580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.176123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278e912-4906-41f0-93e7-6b4fb700cb0f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.214339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.215279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.215766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.216229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.217085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:10.217566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.219099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.231348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 334/667] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:10.232361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.238265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18738921-1066-4661-918d-d0f559d0afe0 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.273976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.274635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.274913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.275192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.286161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:10.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:10.292505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:10.293331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 334/668] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:10.293995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.347433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739563e6-419f-4878-ab2f-a774621c22e6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.381686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.387718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.388332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.388631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.388976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.401216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.408815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68099a-e2da-4323-b51b-437f477d655c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.447126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.447878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.448195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.448424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.449164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:10.449491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.450950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.462523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.464241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.464457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 335/669] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:10.469240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.578304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 335/670] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:10.579490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.592692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d642a-f946-471d-a049-9573d3d7fb80 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.613050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.631055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.631765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.632057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.632354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.639582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.641156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:10.654288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:10.655378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:10.656267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 336/671] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:10.656909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.673434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.690925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25cb12-9561-4bc8-8d57-f06aeb5090e7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.725817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.726431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.726712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.726987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.727690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:10.727995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.729386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.742062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 336/672] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:10.742062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.770851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f3a6a-e414-4da3-b142-b93fdd1cc184 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.807453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.808067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.808367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.808654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.809445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=49153b6e04e24c8d93d323d0a4757684) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:10.809878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.811434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.818339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 337/673] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/role_assignments?user.id=49153b6e04e24c8d93d323d0a4757684&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:10.819125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.849024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.862568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110c6ff7-2b56-447f-90fe-389c891b2fec None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:10.913974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:10.915001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:10.915492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:10.915993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:10.924442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:10.930238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:10.932688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:10.948646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:10.949622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 337/674] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:10.951237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:10.959485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533f38e-1ebc-42fa-907e-014f6045ee5a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.001885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.002514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.002793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.004743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.004743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:11.004743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.005934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fd00a-b091-4292-8a27-8b1025b5db33 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.021468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 338/675] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:11.022739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e861-e109-4fbe-94be-89cdeb7c7fe8 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.023026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e861-e109-4fbe-94be-89cdeb7c7fe8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.023303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470e861-e109-4fbe-94be-89cdeb7c7fe8 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.028379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 339/676] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:11.028379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470e861-e109-4fbe-94be-89cdeb7c7fe8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.075600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760af0-2fef-4ce4-90c3-d7a3b93a817a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.114666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.115333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.115613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.115886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.129283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.131304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.134509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:11.148995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:11.150071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:11.150978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:11.150978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:11.150978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:11.150978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:11.150978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 338/677] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:11.151442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.271583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00858e-24df-4931-8034-a9dc1f1ae5f8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.273000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470e861-e109-4fbe-94be-89cdeb7c7fe8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.310955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.311667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.311956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.312231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.312828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.313030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:11.313278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.313452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.313709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.313980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.314647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.314742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=f260187ace2043849cb3a4ab780899a9) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:11.315045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.316761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f5c5b-0960-449f-89bd-c31999f3aada None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.321497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.325573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 339/678] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/role_assignments?user.id=f260187ace2043849cb3a4ab780899a9&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:11.327054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.327337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.327610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.343804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 340/679] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:11.345174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.355050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:11.356442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:11.434553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.449583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 340/680] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:11.450854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.490736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d233f-290f-4d01-9b93-909afdfef2b3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.535616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.536387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.536773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.537143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.537845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:11.538144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.539675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.555927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 341/681] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:11.556887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.591388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4390a5-d462-4cdc-84a5-c2df5fa3b760 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.628041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.635037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.635686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.635962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.636231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.645251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.646718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:11.663422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:11.664328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:11.665263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:11.665775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 341/682] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:11.669016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.681191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fcb4f0-e6df-4183-ad08-7471bbbb3e0b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.724449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.725347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.726093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.726546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.740397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:11.747659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:11.748911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:11.749918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:11.752454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 342/683] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:11.752682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.754396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.801045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49a3d-e6c9-4a24-a2a8-f2311724e62d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.840372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.841288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.841736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.842162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.850931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.852740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:11.866074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:11.867228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:11.868373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:11.870785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 342/684] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:11.871690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.873375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec8ce-88b8-4985-a806-0ee5ae5fc45c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.892042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:11.910730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:11.911488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:11.911863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:11.912230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:11.913119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:11.913523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:11.915050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:11.927798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 343/685] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:11.928898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:11.986303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae4cb6-be3c-4745-bb5b-30246059285f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:11.999827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.037290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.037510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1d83d-a31a-478a-bfd2-d3dae2670481 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.038295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.038720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.039189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.041297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:12.041767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.043296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.055776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 343/686] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:12.056832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.080742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.081594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.082889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.083264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.084162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:12.084587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.086299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.109655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 344/687] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:42:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:12.110719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.158042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.188041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c37921-b804-41ea-a9b8-ff5fe0a9d79d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.230245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.231003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.231381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.231749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.232712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=9ef5ca1eeba741259394f0ab37dd3751) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:12.233113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.234621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.240068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fef0a-81c3-40e2-a68d-2eec7bbe0298 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.241216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 344/688] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/role_assignments?user.id=9ef5ca1eeba741259394f0ab37dd3751&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:12.242198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.283289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.294472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.295649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.296082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.296531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.312594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.315000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:12.337016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:12.338137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:12.339215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:12.339215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:12.339215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:12.339215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:12.342367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 345/689] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:12.351099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.395776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c58f5-c251-4d9b-b379-056d58417868 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.463313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.463936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.464220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.464571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.490020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.508524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7f702-d702-4f49-ad6e-d8daa43613b8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.549091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.551085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.556428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.556914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.557198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.557477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.558179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:12.558487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.559873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.579287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 346/690] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:12.587966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.596839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.670552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 345/691] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:12.679263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.714960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add36c-a2df-4945-8130-65be9db5abab None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.787322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.787978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.788256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.788554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] PATH_INFO: `/v3/users/nova` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.803925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.805416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:12.829103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:12.830264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:12.831447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:12.834996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 347/692] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:12.840534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:12.856697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.871476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363baf49-6edf-4094-9cfe-87c6d21aec97 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:12.943334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:12.943981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.944262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.944559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.949328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:12.962266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:12.963746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-940eb90a-74cf-4c21-bb1c-c624c0f9ece7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:12.980018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:12.980986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:12.981923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:12.981923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:12.981923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:12.981923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:12.981923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 346/693] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:12.983013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7780-14b8-4caf-b438-47071fc6986a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:12.983292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7780-14b8-4caf-b438-47071fc6986a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:12.983566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e7780-14b8-4caf-b438-47071fc6986a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:12.989457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 347/694] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:12.989457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e7780-14b8-4caf-b438-47071fc6986a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.007742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899014c8-dc3d-4fd0-9bf1-0f6aa3ca479e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.051974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.053543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.054246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.054696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.055914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:13.056800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.058920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea18a9a-175c-454f-bb70-76b2359c1499 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.079283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 348/695] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:13.080770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.081115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.082072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.100814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:13.102291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:13.108014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e7780-14b8-4caf-b438-47071fc6986a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.147266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.148144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.148683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.149245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.150165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:13.150731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.152327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.166267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 348/696] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:13.167466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.180031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.196359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 349/697] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:13.196359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.242860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.311659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ac2d4-48d0-440b-8664-5d4c819c4887 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.314303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d2865d-12ad-477e-8b61-25d043a96bc5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.349339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.350429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.350931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.351417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.358815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.359439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.359720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.359992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.360308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.362223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:13.382506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:13.383448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:13.384401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 350/698] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:13.384897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.448290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.450103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.452278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.496254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f046a8b-fb5d-490a-952b-9bd8808a55fb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.505965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.536428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.537239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.537630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.538487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.538818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.540427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:13.552825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:13.554031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:13.555059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:13.557054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 351/699] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:13.558057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.574029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 349/700] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:13.575192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.667151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f040c3c-30de-4229-8f10-fe97d7f417bc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.685353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b7ae-71ff-4dae-9a1f-a1c859d80be5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.689802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.707254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.708047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.708579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.708965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.710051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:13.710334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.711884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.727546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 352/701] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:13.728672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.739686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.740642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.741180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.741686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.742737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:13.743352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.745051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.751115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 350/702] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:13.752267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.768424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:13.837021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86708bd-b8c3-4318-8b14-e90a27dd033b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.866877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a94cac-709a-4af3-a200-e3f317f171d2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:13.872006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.872681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.872965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.873238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.883326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.884810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:13.899995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:13.901180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 353/703] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:13.902099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.908150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:13.908809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.909093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.909369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.910606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=352be9523cf1454a86c16640a87ece9c) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:13.910920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:13.912538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63aede8-288a-4792-9ab2-29a827be4495 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 351/704] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/role_assignments?user.id=352be9523cf1454a86c16640a87ece9c&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8e8a-c5f7-49ad-bb6f-563d4170edef None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8e8a-c5f7-49ad-bb6f-563d4170edef None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab8e8a-c5f7-49ad-bb6f-563d4170edef None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 352/705] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:13.928746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8e8a-c5f7-49ad-bb6f-563d4170edef None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:13.948062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.038388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.051016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d2896-0175-4d73-86c9-d225aabfdc8d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.073550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab8e8a-c5f7-49ad-bb6f-563d4170edef None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.108029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.109278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.109714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.110205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.111276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:14.111746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.116548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2726a54f-8a05-42b7-a3f5-9f8744d15a93 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.131405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 354/706] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:14.132765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.133116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.133436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.133826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.134493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.134983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.135464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.136506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:14.137070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.139185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.159732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:14.161867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:14.165134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 353/707] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:14.167381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.261549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 355/708] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:14.291236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.324463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bd314-f1ac-4526-b059-a42cc6376f3b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.331733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.375871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.376567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.376853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.377128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.387550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.401528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.403371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:14.424081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:14.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:14.427019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 354/709] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:14.432951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.447693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52967c-c8cf-4b46-afb2-873df1396d9f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.491791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.492526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.492809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.493086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.516464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.516464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:14.535942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:14.537549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:14.538477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:14.538477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:14.538477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:14.538477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:14.538477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 356/710] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:14.544477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.612520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30ce66-1fa4-4514-aa6e-0f5f87a71725 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.661394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.662017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.662299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.662574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.663327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:14.663631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.665256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.668171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.681147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 355/711] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:14.688909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.717853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ccbed-855c-4765-be30-a6d7eccd1c73 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.767149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.767698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.767977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.768252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.769576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:14.770120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.772271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.786543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:14.791642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 357/712] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:14.798517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:14.871909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62268f7-d45e-4883-8503-e8bc9fd756cf None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.913044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ec4fe-6d19-4fa1-bc8b-c40692965bfc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:14.922499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.923123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.923853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.924134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.925405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:14.925721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.927698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8383c240-4507-4760-be13-eb37bc924dd3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.938637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 356/713] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:14 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:14.940679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed9f30-1b43-459f-85b2-ed40283192aa None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.941529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed9f30-1b43-459f-85b2-ed40283192aa None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.941815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed9f30-1b43-459f-85b2-ed40283192aa None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.948398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 357/714] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:14.950273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.950564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.951373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.958120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:14.958878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:14.959163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:14.959436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:14.960150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:14.966853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:14.981207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:14.982428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:14.989734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:15.020632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 358/715] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:42:14 2022] POST /identity/v3/users => generated 254 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:42:15.118849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 358/716] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:15.154657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:15.290949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37359a00-07b1-425d-8485-a54b6645a3a3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:15.329811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:15.330464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:15.330747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:15.331023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:15.331932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:15.333445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:15.343692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:15.344734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:15.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 359/717] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:15 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:15.351335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:15.409045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:15.463641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0995eb-b484-4e67-85f6-a6d3dc885a37 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:15.499783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:15.500424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:15.500702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:15.500967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:15.501640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] RBAC: Adding query filter params (name=network) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:15.501937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:15.503306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994622f-6142-4d6d-ac44-c7f64483538c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:15.512444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 359/718] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:15.516962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:15.586462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:15.638533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42fbec9-bb65-41f5-a4e6-f6cf27adf2c6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:15.695004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:15.695627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:15.695907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:15.696180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:15.696908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] RBAC: Adding query filter params (type=network) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:15.697206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:15.698567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98320628-bda5-4a48-90ee-b89a8ee0aaa9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:15.708478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 360/719] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:15 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:16.654035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616268-a394-494d-bf16-d122482be890 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:16.654550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616268-a394-494d-bf16-d122482be890 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:16.654550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e616268-a394-494d-bf16-d122482be890 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:16.656136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 360/720] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:16.661881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:16.662359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:16.662435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:16.689678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:16.691633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:16.731290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753d72-bca6-4d82-be77-823df1ebbd84 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:16.731573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753d72-bca6-4d82-be77-823df1ebbd84 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:16.731856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e753d72-bca6-4d82-be77-823df1ebbd84 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:16.734936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 361/721] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:16.746929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:16.747205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:16.747473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:16.778464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:16.779562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:16.793292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 361/722] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:16.828509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:16.908363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 362/723] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:42:16.928063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:16.978917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:16.988603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bc0ade-7356-43b6-98b9-5670a6735613 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.080706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.080896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:17.081254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.081651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.082010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.083019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:17.084513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6242a2fe-544a-44cd-a85f-a1c8cca94a3f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:17.116193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 362/724] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:16 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:17.173073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0a184-d7cc-46a2-8619-891e3e7ce6b2 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.246643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.247400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.247681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.247953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.256065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf950b-a30c-43da-841f-725054ad7630 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.256348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf950b-a30c-43da-841f-725054ad7630 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.256652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cf950b-a30c-43da-841f-725054ad7630 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.258267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 363/725] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:17.261304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:17.264023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.264396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.264678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:17.270095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:17.271067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:17.271947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:17.275191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 363/726] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:17.281449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.310280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:17.314588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:17.422891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 364/727] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 20:42:17.445676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:17.450387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33da3bbf-e0a9-4901-9906-dce46211341b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.455983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.496179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.496827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.497110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.497380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.498102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:17.498406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:17.499796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:17.504530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:17.511112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 364/728] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:17.516392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.569133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916ce3d8-2987-4589-abee-1af9a5c6983a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.608466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.609257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.609604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.609944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.621324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:17.626422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:17.627372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:17.628207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:17.632469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 365/729] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:17.634524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.642002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9329ea-50be-4858-a087-6344e3c6c87b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.681207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.682625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.683122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.683594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.695865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:17.697823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:17.714539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:17.715474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:17.716330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:17.716330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:17.716330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:17.716330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:17.718563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 365/730] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:17.724255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.744040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:17.842341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508af5db-4c4a-4253-a149-2d6f92a65e35 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.855171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351594b-b5ea-4abf-ae2b-9fd5eb77c476 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:17.856882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:17.883432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.884458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.884952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.885449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.886448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:17.886996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:17.888603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:17.903239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 366/731] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:17.909145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:17.911340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:17.911970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:17.912255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:17.912559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:17.913285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:17.913591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:17.914976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:17.934843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 366/732] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:17.941763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.048878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.078472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c28e3-d80f-4af5-ba9d-4f6a783b745a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.110332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff709eac-2dbc-4338-9f09-b39d6cd1f97f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.117825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.118765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.119265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.119744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.127919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.130345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ed4d3aa5-7dfc-4e3c-8919-74b92e7b4cf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:18.142548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:18.143504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:18.145036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:18.147716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 367/733] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:18.149162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bb1c-25ff-464e-8c08-4816bfa5a783 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.149668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bb1c-25ff-464e-8c08-4816bfa5a783 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.150143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b697bb1c-25ff-464e-8c08-4816bfa5a783 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.155735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 368/734] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:18.155735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697bb1c-25ff-464e-8c08-4816bfa5a783 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.155942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.156893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.157071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.157353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.176264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.178054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-cecce55c-4f6b-4976-b368-4862222e63ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:18.198347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:18.199250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:18.200130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:18.200130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:18.200130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:18.200130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:18.200130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 367/735] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:18.200956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.202079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.202399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.222625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:18.224280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:18.236963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.293359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b697bb1c-25ff-464e-8c08-4816bfa5a783 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.308621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 368/736] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:18.309962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.346749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.347927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.348221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.348520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.349322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:18.349628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.351061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.353195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.372842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 369/737] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:18.372842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.426840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ae6e1f-af72-4e2f-8d5c-dde084e5e1c7 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.463920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.464817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.465273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.465551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.466247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:18.466554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.467924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fe74a-4e14-4eb8-bc79-e6ef20b68987 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 369/738] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805cee-63eb-4e27-8ca6-58e128490933 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805cee-63eb-4e27-8ca6-58e128490933 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b805cee-63eb-4e27-8ca6-58e128490933 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 370/739] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:18.484538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b805cee-63eb-4e27-8ca6-58e128490933 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.492543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec935c1-49e3-462e-9762-2aafa3bf314b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.494435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.547963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.548759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.549041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.549312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.560703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:18.566448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:18.567551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:18.568537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 370/740] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:18.569230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.608519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b805cee-63eb-4e27-8ca6-58e128490933 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.631626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.647604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.648651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.649152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.649490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.687841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07fb15-e648-4eda-9d3a-8e6418d099d6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.726397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.727766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.728224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:18.728898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.729180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.729454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.737166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.738814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:18.740437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-dc4d0d21-d578-4961-88ce-1c28514cfc43 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:18.752395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:18.753471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 371/741] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:18.754833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:18.755466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:18.755466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:18.774867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:18.776557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:18.853890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 371/742] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 20:42:18.854616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.872363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 372/743] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 20:42:18.872363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:18.915034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:18.981756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd047f2-bc34-4ce3-908a-09e345129334 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.987925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a85f2e-3291-4dca-8901-1f9872263da7 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:18.990599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.021416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.022046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.022332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.022611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.023351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:19.023633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.024617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.025062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.025370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.025716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.026053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.026913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:19.027288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.028744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.034316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 372/744] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:19.040173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.054833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 373/745] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:19.054833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.165477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb54a24-c3f6-49d1-bbf1-6b8a81853d15 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.170449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.173555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40755f-f55c-4975-9748-4b730c27b279 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.209650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.210304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.210588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.210866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.215150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.215774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.216051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.216320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.217034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.218408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.222773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.224508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.246492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 374/746] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:19 2022] POST /identity/v3/services => generated 240 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:19.246492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:19.254926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:19.255941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:19.256852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:19.256852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:19.256852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.256852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:19.264576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 373/747] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:19.265452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.295238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.395172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0baa3ea-2fb1-4121-832a-bd8985f3084a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.402152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16fa51-10ff-4726-824d-c1177e387b90 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.438113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.438775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.439086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.439365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.452157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.453707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.454674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.455333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.455824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.456114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.471821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:19.473828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:19.475397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:19.477865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:19.477865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:19.477865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.477865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:19.478170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.478256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 374/748] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:19.478256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:19.499822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:19.500804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:19.501665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:19.501665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:19.501665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:19.501665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:19.501665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 375/749] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:19.502592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.525004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.600447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.638303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9328c-9b17-4cf3-a326-bae10f6438c7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.689749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.690392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.690746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.691306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.692148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:19.692507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.693974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.708764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 375/750] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:19.709490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.804896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775fa45-beab-4bef-83ca-43c1a1243b69 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.842074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a04b2-7364-4f22-8d89-3dd321dd00af None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:19.857876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.858507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.858793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.859167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.860075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:19.860404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.861827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b29d43-da64-4951-9ee2-a82cd88f436e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.874264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:19.876252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 376/751] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:19.877738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668680eb-1062-4e73-a46f-c7ebc65c4de1 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.878026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668680eb-1062-4e73-a46f-c7ebc65c4de1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.878865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668680eb-1062-4e73-a46f-c7ebc65c4de1 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.884131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 377/752] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:19.884131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668680eb-1062-4e73-a46f-c7ebc65c4de1 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.888830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:19.889468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:19.889748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:19.890021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:19.890721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:19.891484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:19.892919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:19.906725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 376/753] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:19.911168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:19.961337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.002136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668680eb-1062-4e73-a46f-c7ebc65c4de1 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.026964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e4602-5fbc-4767-83a0-f100d3c4a51e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.043910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.044578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.044859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.045128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.053403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.054848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.065527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.066204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.066780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.066780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-07a1ce21-0fe7-4a61-beca-f3c74c9951fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:20.066943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:20.067743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:20.068698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 378/754] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:20.069723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.069824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.070039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.078832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.080286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.090202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:20.091428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:20.092462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:20.093372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:20.094259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 377/755] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:20.094733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.154949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.176904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 379/756] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:20.176904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.218084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57afaf-96d5-418b-8a30-368b153ebdbe None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.237236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.255780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.256456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.256742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.257022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.264804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.266251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:20.279235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:20.280543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 378/757] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:20.281515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.289376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b256a69-d040-41d2-9e60-4e1a9f3c9913 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.324622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.325156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.325433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.325703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.326450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:20.326751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.328179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.339977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 380/758] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:20.340623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.393479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f593a35-e6e8-42ef-935b-f81c7734c327 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.407428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.430230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.430848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.431129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.431401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.432142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:20.432477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.433898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.448239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 379/759] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:20.449180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.460102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28308fc6-72d0-4d73-b42c-35594f28f9a2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.477153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.497027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.497756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.498037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.498312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.498941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.500313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:20.512524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:20.513489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:20.514323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:20.518637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 381/760] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:20.519595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.566548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5c307-1d83-4082-9e28-b68d4110997c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.606184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.607030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.607466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.607897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.608780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:20.609097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.610536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.630146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65584fb5-2fd6-4cc8-a9d9-3a97dd30a53e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.636104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 380/761] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:20.636104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.667546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.668190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.668585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.668743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.669529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=352be9523cf1454a86c16640a87ece9c) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:20.669915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.671424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.673550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.685887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 382/762] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/role_assignments?user.id=352be9523cf1454a86c16640a87ece9c&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:20.685887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.760818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:20.782524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a889f27b-ec69-48ef-a7b0-06720ec2b0af None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.829952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.830893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.831194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.832616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.835431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=02a04385570944ecbcaf96003f5c5753, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.837231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.837994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41017f-fcd2-4873-b1a4-34657f8dac57 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:20.860187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 381/763] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:20 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:20.862098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:20.921561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:20.923327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:20.924466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:20.925320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:20.930024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:20.930024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:20.930382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:20.947780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 383/764] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:20.952088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.038123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:21.053031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f5612-271e-4d9c-8815-c3a0a2acde9b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.094690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.095805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.096325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.096831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.117683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd4023-7cdb-4526-8d5a-3db29cf3a921 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.131375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:21.169211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.169961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.170249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.170553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.171278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:21.171592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.173015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.178585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.180629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.181161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 384/765] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:21.188006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.279649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:21.302948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 382/766] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:42:21.310870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.360679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9458-e3bc-4471-9ca0-bb6640fabed3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.423851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.424507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.426016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.426275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.427032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] RBAC: Adding query filter params (interface=public, service_id=3676d0479cda45d69cd03c64e51a20d8, region_id=RegionOne) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:21.427617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.431183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.442245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 385/767] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/endpoints?service_id=3676d0479cda45d69cd03c64e51a20d8&interface=public®ion_id=RegionOne => generated 194 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:21.448728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.462648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467a166-c761-4ef6-aef7-97f4981dd24c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.508720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:21.549018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.549672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.549955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.550296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.569173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.571058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-402a2be0-3774-4a97-9e39-fe6fde549103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:21.588954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:21.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:21.590808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:21.590808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:21.590808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:21.590808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:21.592848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 383/768] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:21.594813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1e41-946f-44b0-a38f-f7e020b48ca2 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.595346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1e41-946f-44b0-a38f-f7e020b48ca2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.595824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a1e41-946f-44b0-a38f-f7e020b48ca2 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.602583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 384/769] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:21.602583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1e41-946f-44b0-a38f-f7e020b48ca2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.627241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27e002-c04d-4650-99db-dde1ef92ba6a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.663923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:21.682963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.684207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.684747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.685454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.686886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.688823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eef3932-4754-4d63-8c18-4dec88fd0e32 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.697719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 386/770] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/services => generated 772 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:21.699205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.699639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.700041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.724185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:21.726549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:21.783562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a1e41-946f-44b0-a38f-f7e020b48ca2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:21.832560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 387/771] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:21.853796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:21.854605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:21.854996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:21.855374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:21.856202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:21.856644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:21.858161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:21.876365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 385/772] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:21.883335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.898089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:21.949625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.041422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d3de9-a279-4040-8839-6f3c7fe2952d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.062609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be37b48-4b7f-4696-a325-4bc4b256b9ad None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.065486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.083147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.083818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.084103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.084407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.092919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.094204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:22.108728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:22.109685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:22.110982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 388/773] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:22.117820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.119256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.121285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.122207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.122728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] PATH_INFO: `/v3/services/network` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.123622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.125296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:22.137735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:22.138803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:22.139604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:22.142626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 386/774] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:21 2022] GET /identity/v3/services/network => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:22.155830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.275730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab06905-cd67-43d4-b56f-354cdf9654ad None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.297869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.315540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237f0dd-d943-436e-ba7d-16b6016d531d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.319436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.320426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.320907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.321402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.322274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] RBAC: Adding query filter params (name=network) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:22.322808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.324517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.330071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 387/775] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:22.344166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.363303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.363896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.364180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.364503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.365397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:22.365661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.367115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.381431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 389/776] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:22.386561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.392368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.539637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49993e-826a-4f76-9482-b36353bd292a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.596797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50fa8-70bb-42c4-80d7-f3055353da6a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.604426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.605053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.606014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.606014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.606295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] RBAC: Adding query filter params (type=network) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:22.606543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.607911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.614703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 388/777] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:22.621486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.663339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.664358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.664856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.665327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.666384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=1fb00a6bca4f48468574ff17083e1fbf) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:22.666912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.668768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.680940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 390/778] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/role_assignments?user.id=1fb00a6bca4f48468574ff17083e1fbf&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:22.727087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.733720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd740364-9d8d-41a8-a8db-8fb0cc4a8664 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.772231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.772907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.773189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.773463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.774234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] RBAC: Adding query filter params (interface=public, service_id=15280c2bcccc459d8b80c43a86d06b0a, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:22.774539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.775957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c96e92-49f1-4e89-835d-7e1277483128 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.785242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 389/779] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/endpoints?service_id=15280c2bcccc459d8b80c43a86d06b0a&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:22.790987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:22.822361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:22.918335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad024cc-67af-4b0a-a838-dbfe2b58c25f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:22.954570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:22.955550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:22.956035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:22.957080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:22.958047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:22.959684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b4df52-c3e1-4878-b297-58f84de837ca None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:22.965865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 391/780] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:22 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:23.130629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963fe4-2317-4468-817c-302673d542bc None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.131618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963fe4-2317-4468-817c-302673d542bc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.132124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93963fe4-2317-4468-817c-302673d542bc None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.134007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 390/781] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:23.141265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb4e4-c864-4fee-8223-2f87c2d5e3e8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.141801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb4e4-c864-4fee-8223-2f87c2d5e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.142279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbb4e4-c864-4fee-8223-2f87c2d5e3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.164947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-01bbb4e4-c864-4fee-8223-2f87c2d5e3e8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:23.166136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-01bbb4e4-c864-4fee-8223-2f87c2d5e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:23.237583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 392/782] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:23.267030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93963fe4-2317-4468-817c-302673d542bc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:23.405774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93963fe4-2317-4468-817c-302673d542bc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:23.450307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:23.450957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.451238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.451511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.459317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c00aa-756e-4be1-8536-3cc44d0b53fd None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.459623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c00aa-756e-4be1-8536-3cc44d0b53fd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.459851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c00aa-756e-4be1-8536-3cc44d0b53fd None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.461201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 393/783] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:23.466409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.466895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.467170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.472385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:23.489791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:23.490773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:23.491594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:23.494074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 391/784] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:23.504386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:23.504386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:23.505874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:23.505985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:23.549248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:23.628472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 394/785] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:23.679812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:23.707313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f61e91-b4fb-463d-97c0-278d1d444fac None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:23.772428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:23.773531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.774017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.774516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.775460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:23.775969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:23.777663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:23.787801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 392/786] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:23.793523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:23.832579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ae723-0381-40a8-adf9-c927d74756c3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:23.869267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:23.869986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:23.870310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:23.870603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:23.881393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:23.886497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:23.887412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:23.888268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:23.891656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:23.892599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 395/787] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:23.905728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:23.979392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.014997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087dff2b-eff0-4822-8703-b03657e2c55e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.054906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.055569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.055851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.056130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.070127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.071757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:24.091046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:24.091989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:24.092905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:24.092905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:24.092905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:24.092905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:24.100497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 393/788] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:24.101367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.147027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabb39d-d39d-46e9-b40a-0e39f8f95f2e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.189463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.190103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.190385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.190820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.192199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:24.192535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.194127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.209042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 396/789] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:24.212275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.288750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.327971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c655ef42-6ab7-46b4-90b2-d654194efb31 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.367409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.368196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.368533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.368816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.370563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3757b55-8a7c-4a70-8457-8be73133f9b3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.381584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.383060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:24.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:24.404223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:24.405120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:24.405120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:24.405120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:24.405120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:24.405348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 397/790] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:24.412039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.443003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.449579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.450287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.450640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.450991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.451763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:24.452156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.453661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890dd252-cb01-452b-9cdb-6e810bc73eaa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.467326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 394/791] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:24.468794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9ea1-7ac4-4921-81ea-39cbcb6ba25c None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.469151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9ea1-7ac4-4921-81ea-39cbcb6ba25c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.469499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af9ea1-7ac4-4921-81ea-39cbcb6ba25c None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.471153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 395/792] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:24.473799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9ea1-7ac4-4921-81ea-39cbcb6ba25c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.535679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123173e-f5c4-4a46-9091-39c2e82fad9b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.560788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.573843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.574568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.574921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.575267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.576052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:24.576464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.577930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd4de6-f4dd-4cdf-aa88-4431b35472a7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.595423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 398/793] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:24.595423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.595423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.595423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.611630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:24.613278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:24.625016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af9ea1-7ac4-4921-81ea-39cbcb6ba25c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.668410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.669613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.669989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.670358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] PATH_INFO: `/v3/users/swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.679848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.681682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.697904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 399/794] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:24.699038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:24.701043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:24.701964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:24.702897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:24.704409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 396/795] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:24.708445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.779794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.808096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f522c7-5c42-4586-b13e-083a17cd3f69 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.811331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:24.845593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f924e0ea-9ad4-45da-9b68-3a203eaf764c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:24.846864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.847567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.847913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.848264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.855619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.857245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:24.869129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:24.870059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:24.870973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:24.875018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 400/796] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:24.876000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.892743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:24.893469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:24.894541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:24.894916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:24.895782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:24.896154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:24.897766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:24.921254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 397/797] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:24.922254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:24.990065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb34c2-5f0b-45a4-87ff-42112c9b9e11 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.027935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.028920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.029463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.029932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.030787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.030943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ceeff-e3c0-4450-b33d-1f2cdfc2d311 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.032610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.042749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:25.044360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:25.045269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 20:42:25.046079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.054602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 401/798] 10.222.0.52 () {56 vars in 1129 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:25.056538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.071519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.072208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.072617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.072996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.073862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:25.074357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.076089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.090057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.103218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 398/799] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:25.104472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.170103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf9708-021e-4db4-8d99-20796d8c8372 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.207192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.207827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.208112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.208420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.214627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d290073-e96e-42f5-ab53-510c88ed6e99 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.219071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.220621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:25.235222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:25.236148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 402/800] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:25.238357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.257974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.258700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.258982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.259252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.259983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:25.260288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.261991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798bee7a-6989-441a-8414-ac87bacdc45e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 399/801] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a301a-4e7d-47ff-aa4a-02d330f3b5f4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a301a-4e7d-47ff-aa4a-02d330f3b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a301a-4e7d-47ff-aa4a-02d330f3b5f4 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 400/802] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:25.271430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a301a-4e7d-47ff-aa4a-02d330f3b5f4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.305783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.334244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.365450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71947bd7-19c4-4e54-bde7-578288d8759d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.393588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a301a-4e7d-47ff-aa4a-02d330f3b5f4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.407421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.408042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.408321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.408626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.431559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.432201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.432507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.432784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.433476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:25.433781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.435166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.448379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 401/803] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:25.448804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.494461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.495921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.551030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.560281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99314a09-0b78-40d3-bb5a-c9228e56292c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.589498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.594745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.595365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.595649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.595967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.596691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:25.597044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.598529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4b016-fa5d-45d9-bda7-35da6b19d96f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.603313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 402/804] 10.222.0.52 () {56 vars in 1124 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:25.604478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bae35-35f6-4493-9f35-cf783a226ab3 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.604765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bae35-35f6-4493-9f35-cf783a226ab3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.605044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bae35-35f6-4493-9f35-cf783a226ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.609341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 403/805] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:42:25.610169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.629716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d03bae35-35f6-4493-9f35-cf783a226ab3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:25.630926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d03bae35-35f6-4493-9f35-cf783a226ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:25.697581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 403/806] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:25.699123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab717f6-9d1a-4d16-bf0c-337d7bd26287 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.699417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab717f6-9d1a-4d16-bf0c-337d7bd26287 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.699696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab717f6-9d1a-4d16-bf0c-337d7bd26287 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.700973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 404/807] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:25.701652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab717f6-9d1a-4d16-bf0c-337d7bd26287 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.721096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481215f1-e629-48d1-867e-895ae095c5d3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.757853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.758489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.758765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.759034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.765637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.767064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:25.777923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:25.778938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 404/808] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:25.779998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.803349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.820578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab717f6-9d1a-4d16-bf0c-337d7bd26287 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.822315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:25.855086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.855629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.855910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.856181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.856912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.858273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890b460-3e2a-4e3a-8ea7-64db32bfcd59 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.880247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 405/809] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:25.881562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.881856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.882135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.882650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06804c32-643f-4d16-88c4-77f7164103eb None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:25.912585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:25.914214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:25.930134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:25.930856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:25.931146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:25.931427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:25.946416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:25.947898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:25.964304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:25.965314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 405/810] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:25.971408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:25.997823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 406/811] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:25.998706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.085807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.096057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.103049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa327ef-0283-4eb5-a666-34452804222f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.134404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c0e7-dc5b-4be2-8b5d-4f28289b73d2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.137583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.138225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.138502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.138769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.149871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:26.155605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:26.156590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 406/812] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:26.158086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.178875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.179968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.180374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.180781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.181635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:26.182049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.183995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.201432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 407/813] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:26.201432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.274301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaa80f3-1d5c-49cd-a232-2abddd65dfb7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.312673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.313286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.313561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.313828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.314598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:26.314800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.316965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.319686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7c7c78-5790-4413-abc6-8161885a4d06 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.329457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 407/814] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:26.330286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.345068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.354502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.360239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.361729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.362012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.362283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.362968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.364603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:26.374036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:26.375216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 20:42:26.376260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:26.380708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 408/815] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:26.381529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.462632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c1056-6381-43ad-9213-40880df38460 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.494598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24a21b-3940-4c47-b40b-31d82882d7e1 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.499218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.499906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.500255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.500614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.501395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:26.501769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.503354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.516565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 408/816] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:26.516565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.544047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.544864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.545219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.545566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/352be9523cf1454a86c16640a87ece9c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.547614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=352be9523cf1454a86c16640a87ece9c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.549142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.572380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 409/817] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:26 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/352be9523cf1454a86c16640a87ece9c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:26.572380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.618660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.662493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1ffd9-28f6-46a4-bfae-aa94cf55010f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.663617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.711145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.711977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.712371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.712824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.721396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.724178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.728125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe086420-3ccc-486b-974a-db21ad579b24 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:26.739936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:26.740978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:26.741843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 409/818] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:26.742810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.768651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.769481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.769871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.770168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.770855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] RBAC: Adding query filter params (name=network) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:26.771161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.772583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.778737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 410/819] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:26.780623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.864978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d4b72-cb14-49d7-8fe2-e23ad44f4257 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.900458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.910177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.910831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.911115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.911389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.924988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067f0a-4f09-4d75-b67b-20780c50fc6d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:26.928494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.934447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.947507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:26.958928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:26.959867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:26.960846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:26.960846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:26.960846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:26.960846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:26.960846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 410/820] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:26.966066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:26.971806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:26.972737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:26.973219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:26.973690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:26.974656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:26.975159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:26.976876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:26.991220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 411/821] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:26.992514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.095334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec96e9f-93fe-4b62-9d13-d4416aa41d6c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.115009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43a57a-a329-4410-a7ce-7cd3ca3241d2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.136869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.137451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.137733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.138041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.138786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] RBAC: Adding query filter params (type=network) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:27.139096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.140566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.151472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 411/822] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:27.152478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.154016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.154673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.154957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.155348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.156147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:27.156531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.158285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.171809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 412/823] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:27.172902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.204170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:27.222106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:27.264848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747249d8-8b91-4c2d-979b-548ea1e033c9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.287317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e25d03-1aa0-45b7-ba47-9d55d104bf84 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.301969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2e955-3605-4563-b973-a1700239087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.302858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e955-3605-4563-b973-a1700239087a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.303145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e955-3605-4563-b973-a1700239087a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.303422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2e955-3605-4563-b973-a1700239087a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.304312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2e955-3605-4563-b973-a1700239087a None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:27.304657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2e955-3605-4563-b973-a1700239087a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.306193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2e955-3605-4563-b973-a1700239087a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.316389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 412/824] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:27 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:27.316389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2e955-3605-4563-b973-a1700239087a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.348690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.349523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.349942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.350223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.350957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.352592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.384700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 413/825] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:27 2022] POST /identity/v3/endpoints => generated 337 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:27.497364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2e955-3605-4563-b973-a1700239087a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.523321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:27.559818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.560514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.560807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.561076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.572854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.574386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:27.587889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:27.588933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:27.589867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 413/826] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:27.594968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.704638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:27.714812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2327a14-0420-4ea8-ae19-108f5bd59375 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.754735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.755288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.755581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.755854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.756752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:27.757062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.758657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.778961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 414/827] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:27.784954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.915098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca8503-f0b8-40d4-97ca-fd87491e4301 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:27.951664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:27.952315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.952626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.952904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:27.964691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:27.966172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-2bffca86-fe2f-4522-a18c-ecb8428cf82a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:27.982402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:27.983328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:27.984189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:27.984189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:27.984189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:27.984189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:27.984189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 414/828] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:27.990004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:27.999055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878f2d-e636-48df-b28f-666c89aabe0b None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:27.999693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878f2d-e636-48df-b28f-666c89aabe0b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:27.999970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45878f2d-e636-48df-b28f-666c89aabe0b None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.001598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 415/829] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:28.006822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.007065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.007305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.025750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:28.027014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:28.042815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.116616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 416/830] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:28.122764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf37fee-d349-4917-830f-a255fe163835 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.134710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.162496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.163178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.163900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.164183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.165193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:28.165528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.167817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.182011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 415/831] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:28.188566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.205882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.265621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ebe5-4f66-42fa-b39f-0d42363977e6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.306408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90e475-0dd5-4b41-bda0-5ce1a0f3649a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.310338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.311511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.311933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.312366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.313123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.314624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:28.325022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:28.326201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:28.327203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:28.329539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 417/832] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:28.335830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.345981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.347011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.347522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.347994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.355593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.357335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.369192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:28.370371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:28.371256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:28.372240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:28.379293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 416/833] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:28.380387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.460662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3aa35-354a-4010-9687-6c32cbe9d38e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.490097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.499784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.500537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.501134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.501134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.501807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:28.502118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.504323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.515761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 418/834] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:28.515761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.539812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc876372-b90d-4489-94a3-44435e37307b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.587494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.589074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.589560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.590295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.591484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:28.592691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.593991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.602999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.617581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 417/835] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:28.618298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:28.636620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6178de9-ba23-44b0-b8ae-bbff2d36c3e9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.674255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.675429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.675766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.676047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.677021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:28.677380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.678696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97e3ab-794f-4397-8e4d-3808b29c0565 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.692633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 419/836] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:28.747072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c06db1-7718-4478-85db-3fd171334021 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:28.763526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622e364-3516-4a9d-b402-dd3971d25aec None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.763844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622e364-3516-4a9d-b402-dd3971d25aec None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.764083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622e364-3516-4a9d-b402-dd3971d25aec None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.765860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 420/837] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:28.771310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.771620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.771861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.791449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:28.792601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:28.792754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:28.793632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:28.794111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:28.794906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/1fb00a6bca4f48468574ff17083e1fbf/roles/78496973911641848ac1ee3031c692e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:28.798804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:28.819950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=1fb00a6bca4f48468574ff17083e1fbf, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:28.822434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:28.852473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 418/838] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:28 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/1fb00a6bca4f48468574ff17083e1fbf/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:42:28.882732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 421/839] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:28.906905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.021680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73ec12d-1e49-4d07-824a-e8e26f40f783 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.062127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.062779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.063068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.063347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.074684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:29.080238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:29.081711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:29.083008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:29.086936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 419/840] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:29.087979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.166152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:29.201742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f02ad-6756-4b2b-bf98-edd6785212b7 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.239831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.240484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.240769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.241042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.241764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:29.242067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:29.243434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:29.256473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 422/841] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:29.258583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.308952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:29.369983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8983cd-bfef-42c7-a6f8-5942b06222be None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.412737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.413756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.414235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.414729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.425875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:29.427889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:29.445917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:29.446931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:29.447784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:29.447784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:29.447784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:29.447784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:29.450080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 420/842] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:29.460363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.567572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e579e4c-5004-4d9f-9b59-35cb5a46fe4a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.599171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:29.604955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.605769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.606053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.606332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.607033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:29.607341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:29.608757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:29.626437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 423/843] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:29.627775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.734611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:29.742908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d93186-59fe-4a49-806f-a86197c23b7d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.779039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688939d-3e85-4475-947b-1564267c373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.779680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688939d-3e85-4475-947b-1564267c373f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.779963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688939d-3e85-4475-947b-1564267c373f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.780277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688939d-3e85-4475-947b-1564267c373f None admin] PATH_INFO: `/v3/users/swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.789000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688939d-3e85-4475-947b-1564267c373f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:29.790440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688939d-3e85-4475-947b-1564267c373f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-d688939d-3e85-4475-947b-1564267c373f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:29.802721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:29.803701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:29.804800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 421/844] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:29.811633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:29.901093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a881-0e16-4433-9814-0c1849f55be3 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.901280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a881-0e16-4433-9814-0c1849f55be3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.901539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47a881-0e16-4433-9814-0c1849f55be3 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.903738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 422/845] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:29.910187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.910514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.910787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.922568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65511c6-796a-4e4e-bf17-8a6254d96c0c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:29.929837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:29.931324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:29.960150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:29.961058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:29.961336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:29.961605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:29.962393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:29.962696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:29.964115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:29.989462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 424/846] 10.222.0.52 () {56 vars in 1135 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:29.995424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.024493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 423/847] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:30.046540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.061228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.120032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad926eb-1cfe-4291-a856-bf146e3b7e99 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.157316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.158152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.158265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.158460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.170022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.193036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4ebed-7e3b-4364-b268-c14a2c3df9ce None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.238540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.238775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.239232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.239510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.239779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.240427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.241425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.241778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:30.252824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:30.255215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:30.256328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 424/848] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:30.260486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.320168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.372312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35d6d8-e313-432d-9813-99af2b07d806 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.394340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 425/849] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:29 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 20:42:30.399607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.414875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.416322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.416933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.417400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.418509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:30.419352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.420932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.429582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 425/850] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:30.434401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.459361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.523740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95007e2a-939f-46df-a995-404f3e1e8f6a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.558607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.559478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.559758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.560029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.570968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.572443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.576660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75456402-47ad-4cb7-92d5-a8718a27d947 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:30.589379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:30.590296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:30.591166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:30.591166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:30.591166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:30.591166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:30.595390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 426/851] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:30.596325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.600220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.620766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.621710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.622212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.622681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.623496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:30.624054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.625673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.636585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 426/852] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:30.713282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16092c-0dea-471a-9ef5-bfea56795fee None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.751810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.752474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.752758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.753122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.753833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:30.754137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.755523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81ab34-cc1b-49d8-bfef-3a970e132d7e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.769881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 427/853] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:30.775591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.845659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223b5c-bcdf-4bfb-87a4-4aff639f6a4c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.846016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223b5c-bcdf-4bfb-87a4-4aff639f6a4c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.846300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c223b5c-bcdf-4bfb-87a4-4aff639f6a4c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.848400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 428/854] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:30.853183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.853471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.853750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.872389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:30.873586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:30.875885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.896907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daaa878-e2ed-4685-8ff0-15856ea33e31 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:30.933235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:30.934287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:30.934777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:30.935258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:30.937571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:30.943748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:30.945509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:30.948419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 429/855] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:30.966304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:30.967192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:30.968090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:30.970333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:30.970748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 427/856] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:30.978170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.091336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39659a-5756-424d-a700-39a6d87261d1 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.099946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4999f-0a51-4953-8bf6-ab5cce4a6c4b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.126865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.127485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.127768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.128209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.128938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:31.130305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24383c7d-101d-45ae-8c28-51c289699d5a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:31.137465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.138126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.138414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.138719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.139548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:31.139834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:31.141342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:31.145890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 430/857] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:30 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:42:31.147118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892975c-e27e-4145-9fe9-d8681fbc7478 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.147413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892975c-e27e-4145-9fe9-d8681fbc7478 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.147692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892975c-e27e-4145-9fe9-d8681fbc7478 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.149072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 431/858] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:31.155036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.155379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.155704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.161974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:31.169679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 428/859] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:31.179675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.186126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:31.187409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:31.245133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:31.284693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 432/860] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:31 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:31.305332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.329005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2ef7a2-7c1c-42fe-b094-8bc2b38b9779 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.373729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.375420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.375953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.376589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.377845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=352be9523cf1454a86c16640a87ece9c) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:31.379044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:31.383077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:31.393301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 429/861] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:31 2022] GET /identity/v3/role_assignments?user.id=352be9523cf1454a86c16640a87ece9c&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:31.433889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c55c0-5827-47d9-8afe-ee0c8325ce4e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.475786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.476474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.476760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.477041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.490145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:31.495421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:31.496373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:42:31.497214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:31.500453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 433/862] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:42:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:31.503777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.590492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:31.632281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7ead-96dc-47f0-a6eb-ca92a396dd69 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.669507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.670133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.670387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.670677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.671431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:31.671736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:31.673154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:31.682728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 430/863] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:42:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:31.688530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.734042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:31.799877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f0116-6f6a-4a84-90c2-d9b88b77f3f2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:31.837437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:31.838116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:31.838398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:31.838677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:31.850911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:31.852416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:31.868641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:31.869587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:31.870542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:31.870542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:31.870542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:31.870542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:31.874473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 434/864] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:31.875832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:31.982810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c2b7-e936-44a6-aae9-75ce1be2b07c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:32.006933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:32.019637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:32.020269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.020583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.020862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.021552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:32.021860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:32.023233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:32.039988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 431/865] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:32.041455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:32.135741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:32.146068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02e22-44b6-4e33-9bc9-547524a261d2 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:32.182413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:32.183070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.183354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.183632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.192198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:32.193697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:32.205319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:32.206314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:32.207248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:32.211477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 435/866] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:32.212892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:32.321015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7942c-4335-40cb-bf53-fccac89e03ed None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:32.362038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:32.363749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.363749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.363749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.364583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:32.364935 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:32.366370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:32.385452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 432/867] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:32.391001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:32.497761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460a446-a1cb-41d7-9ae2-10c8389aedfb None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:32.533222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:32.533851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.534659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.535058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.558363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:32.606697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:32.608152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a95c7-be13-4f56-8751-39dedc9654fa None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:32.721925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 436/868] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:32 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:42:32.723245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:32.775229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:32.803070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4639813-52f3-4ed3-bf37-bb960e1953e8 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.803387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4639813-52f3-4ed3-bf37-bb960e1953e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.803620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4639813-52f3-4ed3-bf37-bb960e1953e8 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.805238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 437/869] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:32.810536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.810788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.811019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.829932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:32.831139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:32.849969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f9f07-d289-4a50-a27e-319ee16e059b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:32.884580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:32.890239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:32.891170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:32.891655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:32.892128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:32.904011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:32.905795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:32.907845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 438/870] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:32 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:32.921448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:32.922335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:32.923190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:32.923190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:32.923190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:32.923190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:32.925876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 433/871] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:32.930581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:32.935406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.044128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8427926-2772-4615-9eb8-b5b2d5b70321 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.046854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:33.056489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b829ec3-ebad-4af7-8957-91209a413d30 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.090349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.091109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.091514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.091887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.092717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.094175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.101390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.102147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.102468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.102710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.103499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:33.104366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.105407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c21945e-890d-4983-992a-e0228beaec7c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.120800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 439/872] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:32 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:33.126945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 434/873] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:33.129118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.184215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:33.263939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78754-fd13-4283-8fb8-3b38efc1fd12 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.303591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.304600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.305023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.305437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.313502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.315127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:33.327417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:33.328320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:33.329991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 440/874] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:33.330579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69df08-3955-4597-b685-8eb8263c0d46 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.330733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69df08-3955-4597-b685-8eb8263c0d46 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.331786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69df08-3955-4597-b685-8eb8263c0d46 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.333761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 435/875] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:33.337665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.339187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.339696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.340179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.363563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:33.364809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:33.454479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:33.462260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b0de-1719-4f7a-bb1a-f6bfae17f4f1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.471020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 436/876] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:33.496048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.511208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.511846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.512232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.512597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.513362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:33.513670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.515125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.530903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 441/877] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:33.538641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.611470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:33.629700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b88fb9-7201-4387-9352-cc595e8ce27a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.668399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a97d18-dde8-4892-8054-07ef48e239df None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.678045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.678741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.679040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.679322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.680117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.681551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0ce463bd-3008-4923-9fe9-9575768ea2ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:33.692439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:33.693460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:33.694273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:33.698611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 437/878] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:33.699748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946c6-261c-401d-b0da-d3ce373393df None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.700036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946c6-261c-401d-b0da-d3ce373393df None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.700314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42946c6-261c-401d-b0da-d3ce373393df None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.701935 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 438/879] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:33.703053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42946c6-261c-401d-b0da-d3ce373393df None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.712211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.712983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.713341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.713691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.714583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=1fb00a6bca4f48468574ff17083e1fbf) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:33.714957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.716538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f52c6a-623a-48c6-af6e-82c9d03e8c84 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.724736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 442/880] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/role_assignments?user.id=1fb00a6bca4f48468574ff17083e1fbf&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:33.724736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.725403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.725403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.752074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:33.753653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:33.770877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:33.829858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42946c6-261c-401d-b0da-d3ce373393df None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:33.834245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 443/881] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:33.856730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.867616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:33.868189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:33.868540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:33.868841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:33.869543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:33.869844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:33.871318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:33.882306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 439/882] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:33.883684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:33.900521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:34.009218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad3a2c-e893-4413-949a-4f8d028e6d19 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.023526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8567105-b3a7-4ee6-9309-661c3ff498b4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.057633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.058257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.058537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.058807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.059427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.060832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:34.071178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:34.072205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:34.073093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 444/883] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:34.075738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.076692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.077180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.077719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.078560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:34.078842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:34.079372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.080961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.093143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 440/884] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:34.099204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:34.129403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:34.199027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:34.203706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185c7a-704e-4b27-87d7-c2e9372285e7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.226089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9bfef2-af60-413d-a335-8866907d51ee None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.247399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.248510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.248999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.249598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.250581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:34.251148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.252792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.259178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 445/885] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:34.265008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:34.265939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.266570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.266849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.267119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.268016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] RBAC: Adding query filter params (interface=public, service_id=26b46031fdbc4caebd641c8aec5a54b2, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:34.268356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.269815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.280418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 441/886] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:34 2022] GET /identity/v3/endpoints?service_id=26b46031fdbc4caebd641c8aec5a54b2&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:34.282952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:34.372994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:34.401199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a75c69-f314-4853-ab4b-03ff923b4c55 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.402680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30e57-0be5-444c-ae7a-d4a0af6484c6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:34.444177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.444913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.445199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.445479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.446161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.447540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a1f3c-38b7-4afe-b5ef-cfc2a431ab0b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.452700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:34.453566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:34.454073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:34.454339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 442/887] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:34 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:42:34.454792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:34.455468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:34.455772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:34.457733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd920e8b-2f9b-461f-b327-ce3529d0c961 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:34.464387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 446/888] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:34 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:35.336647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8dcbb-2df2-4f3e-9327-699a8587ca40 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:35.337166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8dcbb-2df2-4f3e-9327-699a8587ca40 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:35.337242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8dcbb-2df2-4f3e-9327-699a8587ca40 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:35.338871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 443/889] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:35.344214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:35.344675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:35.344847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:35.363718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:35.364956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:35.434629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 447/890] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:35.457129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8dcbb-2df2-4f3e-9327-699a8587ca40 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:35.462364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:35.574777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e8dcbb-2df2-4f3e-9327-699a8587ca40 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:35.612916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:35.613790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:35.613902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:35.614102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:35.614837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:35.616288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:35.626695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:35.627667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:35.629393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 444/891] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Aug 4 20:42:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:35.634235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:35.717045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:35.755666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c2ee7-0c31-4093-a5c2-bbad29afb591 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:35.790170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:35.790865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:35.791156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:35.791423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:35.792092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:35.792414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:35.793766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:35.804127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 448/892] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:35.805128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:35.879030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:35.915631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50abc5-c91f-4ee2-8d6a-b1feec72647b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:35.953014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:35.953638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:35.953921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:35.954197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:35.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:35.955182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:35.956909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbe33e-7b42-45b3-9238-b8f947373cbf None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:35.962589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 445/893] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:35.968170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.007430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d7ec-e1a5-49ce-9367-a117c25168d2 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.007743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d7ec-e1a5-49ce-9367-a117c25168d2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.007994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993d7ec-e1a5-49ce-9367-a117c25168d2 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.009721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 446/894] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:36.015084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.015376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.015657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.036445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:36.037877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:36.077132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bacca-4eef-443b-9808-3beee63f2d92 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.113410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.114564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.115154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.115638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.116643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] RBAC: Adding query filter params (interface=public, service_id=c369442ba4e444768ad7790650117913, region_id=RegionOne) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:36.117147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.118702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.120695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 447/895] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:36.121891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.133420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 449/896] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:35 2022] GET /identity/v3/endpoints?service_id=c369442ba4e444768ad7790650117913&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:36.138558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.145312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.215039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.266933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f3900-7908-4083-a252-484d8eef53cf None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.270224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e50200-c826-46b1-b65b-d069c504cef5 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.306102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.307109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.307585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.308049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.318082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.318709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.318996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.319277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.319937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.321319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:36.321613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:36.326771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:36.327655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:36.328486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:36.329115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 448/897] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:42:36.335335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 450/898] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:36.336780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.455775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0f17a-4795-4975-8abb-cf2cc009ced6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.488804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.498200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.498825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.499110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.499387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.500105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:36.500440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.501817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53660a7f-dcb4-4049-9576-cde8b84a42f7 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.511207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 449/899] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:36.516883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.543932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf185b-4498-4646-9103-7d4445a7cfac None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.544221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf185b-4498-4646-9103-7d4445a7cfac None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.544529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf185b-4498-4646-9103-7d4445a7cfac None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.546226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 450/900] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:36.552031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63be848-b598-4e07-a91d-ef95e2884cbd None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.552694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63be848-b598-4e07-a91d-ef95e2884cbd None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.553050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63be848-b598-4e07-a91d-ef95e2884cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.574214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b63be848-b598-4e07-a91d-ef95e2884cbd None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:36.575421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b63be848-b598-4e07-a91d-ef95e2884cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:36.596266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.649982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd538dc4-d373-4096-a63c-6284dc627b53 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.660583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 451/901] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:36.662061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8907-9f03-4d11-a1ac-9476e7958ece None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.662456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8907-9f03-4d11-a1ac-9476e7958ece None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.662835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e8907-9f03-4d11-a1ac-9476e7958ece None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.664210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 452/902] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:36.669369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.669663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.669941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.689496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:36.691382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:36.693611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.694387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.694678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.694953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.707393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.708845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:36.727288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:36.728211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 451/903] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:36.733836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.776575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 453/904] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:36.776575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.827525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.857112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4624bc-d0af-445a-bb8c-b9785ffc8fd9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.871214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:36.884003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03062f4-abff-446c-aa4b-d01216783562 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:36.895992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.896642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.896927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.897200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.897886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.899258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.915920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 452/905] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:36 2022] POST /identity/v3/services => generated 241 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:36.917300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:36.935411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:36.936457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:36.936944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:36.937491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:36.938411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:36.938975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:36.941470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:36.955194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 454/906] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:36.961253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.068443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9602388-02a1-44e8-a81a-50ecb99b92ec None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.115102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50320a-f0ba-4892-bbc1-4742994a893a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.118047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:37.121866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.122523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.122846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.123153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.124732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.126158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:37.139567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:37.140895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:37.141798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 453/907] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:37.148624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.163285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.164381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.164381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.164599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.174244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.176057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:37.192653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:37.193842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:37.195004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:37.197596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 455/908] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:37.200332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:37.202972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.293791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28f3a2-c03a-4123-9273-79900ae8dd92 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.335635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.336436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.337040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.337309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.338125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:37.338509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.340152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.344015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ba69a-e3db-44c3-9f8b-fa2b13914623 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.351647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 454/909] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:37.353518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.392781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.394114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.394536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.394813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.395619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:37.395923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.398355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.421546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 456/910] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:37.426923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.463874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:37.484433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:37.486694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e28d4d5-512a-4bce-ba39-778acf381863 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.537593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.538210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.538492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.538762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.539507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:37.539915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.541434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.547482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 455/911] 10.222.0.52 () {56 vars in 1114 bytes} [Thu Aug 4 20:42:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:37.559275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.572850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8106d-5800-4615-8e58-193a0ad4dcdc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.634151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.634781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.635063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.635897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.696803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2262f9-3fd6-4403-8885-44502e32d877 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:37.732541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.732888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:37.733320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:37.733744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:37.734022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:37.734195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.734788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:37.736137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:37.763149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:37.769108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 456/912] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:37.862982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 457/913] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:37 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 20:42:37.869641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:37.898014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.000198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43542a-f848-47b8-b99d-641e1cc52978 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.036065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.036737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.037021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.037296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.048354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.050374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:38.065388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:38.066464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:38.067346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:38.067346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:38.067346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:38.067346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:38.068014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 457/914] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:38.073784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.185067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede07dc-9788-4205-809e-4e74e7407cc3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.210148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.224957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.226012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.226510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.226999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.227908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:38.228445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.230050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.243120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 458/915] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:38.249524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.320844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.359813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e557910-fd97-480c-a178-dd74158a89e7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.395498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.396142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.396455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.396727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.404038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.405516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-b9241138-da80-41c5-8e78-9b2ccc359a9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:38.418675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:38.419648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:38.420569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 458/916] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:38.427668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.485774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcdc98-c27e-412b-8f54-6153da24f747 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.486075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcdc98-c27e-412b-8f54-6153da24f747 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.486334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dcdc98-c27e-412b-8f54-6153da24f747 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.488109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 459/917] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:38.494139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.494783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.495273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.519328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:38.521248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:38.547989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863cc37-0ad6-45d3-b49b-37cf685ca8ea None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.588444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.589431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.589913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.590367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.591291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:38.591775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.593410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.604424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 460/918] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:38.615126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 459/919] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:38.621186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.630081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.678260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.691662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.743480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dada5-9dd4-445d-b673-1bbd2f6f18ef None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.744898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a99807b-b3c6-46f1-986e-235902d59677 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.782487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.782982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.783121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.783381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.783659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.784749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] RBAC: Adding query filter params (role.id=c86c3ffc0dd24d849ebd736b1f1af775, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=1fb00a6bca4f48468574ff17083e1fbf) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:38.784862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.785328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.785811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.786243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.786510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.787342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.788960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.796437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 461/920] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/role_assignments?user.id=1fb00a6bca4f48468574ff17083e1fbf&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=c86c3ffc0dd24d849ebd736b1f1af775 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:38.807532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:38.808594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:38.809842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 460/921] 10.222.0.52 () {56 vars in 1132 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:38.818627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:38.936688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e12ca60-e078-4ee8-9964-f1b2c786d02d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:38.963669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:38.978495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:38.979126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:38.979412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:38.979691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:38.980454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:38.980764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:38.982131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447eba3e-94ee-4a83-8f08-08b5b9d8c31f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:38.987985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 462/922] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:38.997805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.092814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbeb32-2824-4152-a722-25db4141593e None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.093349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbeb32-2824-4152-a722-25db4141593e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.093809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbeb32-2824-4152-a722-25db4141593e None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.095287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 463/923] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:39.101010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.101670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.102155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.118813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316d0a3-8c01-45bf-96e7-dc45caa1bbca None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.122682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:39.124118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:39.126181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:39.156079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.156738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.157018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.157320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.157964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:39.158270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.160393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.167306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 461/924] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:39.171703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.224363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 464/925] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:39.241919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.295957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9de0f-ebef-43c3-8103-304ee4e690fa None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.340846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.341473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.341903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.342194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.343204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.344913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.353482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc740f-5d5f-4bd4-b982-708c7083edcb None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.374918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:39.376100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 462/926] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:39.409684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.410664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.411097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.411564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.412464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.414088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:39.425369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:39.426730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:39.427937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:39.430531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 465/927] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:39.439627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.474817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:39.559434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590ea32-b57b-4e06-8012-6789a9bf9e98 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.596019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.597316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.597602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.597880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.598569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:39.598872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.600264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.606003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 463/928] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:39.611627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.721318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397e188-b8db-484e-839d-85e264e7193c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.757253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.757932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.758221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.758493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.759167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:39.759468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.760876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.766449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 466/929] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:39.773425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.869652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:39.881581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c3b0a-9ee0-4ab0-9570-c68e21ee926d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:39.917890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:39.918522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:39.918817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:39.919077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:39.919834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] RBAC: Adding query filter params (interface=public, service_id=c365a22dedc24813ba15d39dc185a5e6, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:39.920145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:39.921547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778bf007-57d8-47f5-9a1f-091c0025b5a1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:39.927518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 464/930] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:39 2022] GET /identity/v3/endpoints?service_id=c365a22dedc24813ba15d39dc185a5e6&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:39.933062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:39.986957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:40.058476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e402b6-95c3-4371-844e-4f3139c55322 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:40.076868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0891de-5d4f-4c00-bbaf-991c9635e8ec None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.077187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0891de-5d4f-4c00-bbaf-991c9635e8ec None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.077421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0891de-5d4f-4c00-bbaf-991c9635e8ec None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.079010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 465/931] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:40.084488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.084769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.085040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.098589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:40.099658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.100127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.100610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.101480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:40.103024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:40.103723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:40.105094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:40.110174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 467/932] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:39 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:42:40.187059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 466/933] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:40.211260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:40.328624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f215160-ab93-4c18-b01a-1587979b1f63 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:40.332495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:40.366699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:40.367510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.367800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.368077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.368801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:40.370406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:40.373130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING keystone.server.flask.common [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 20:42:40.396679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.sql.core [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 20:42:40.396679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 20:42:40.396679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [parameters: {'id': '1a04a99b93ae40ad8e7a5604474d2a33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 20:42:40.396679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86873) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a04a99b93ae40ad8e7a5604474d2a33.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a04a99b93ae40ad8e7a5604474d2a33. Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:42:40.414343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application result.read() Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 20:42:40.415383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application yield resource Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise self.value Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application session.commit() Aug 04 20:42:40.416588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self.session.flush() Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application compat.raise_( Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 20:42:40.417551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 20:42:40.418523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application result.read() Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:42:40.419365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [parameters: {'id': '1a04a99b93ae40ad8e7a5604474d2a33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:40.420192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:40.421057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1a04a99b93ae40ad8e7a5604474d2a33. Aug 04 20:42:40.422208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:40.428677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 468/934] 10.222.0.52 () {60 vars in 1140 bytes} [Thu Aug 4 20:42:40 2022] POST /identity/v3/projects => generated 262 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:40.431567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:40.533269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:40.557046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d002e34-23c3-40ac-9dab-41a589b46f13 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:40.594617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:40.595269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.595547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.595821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.603724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:40.605200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7127b5eb-519e-49c4-9b7a-99d021e878ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:40.617787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:40.618707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:40.619579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 467/935] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:40.625304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:40.733500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de62d9-431c-4fc3-bc92-053e668db9a1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:40.769301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:40.769921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.770199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.770471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.771184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:40.771484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:40.773153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3adbb-772e-46dd-b4ce-b6b6ee7a05ac None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:40.794791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 469/936] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:40.967317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001f8cd-b7f6-44b9-905c-532a17bc4a7d None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.967661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001f8cd-b7f6-44b9-905c-532a17bc4a7d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.967868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001f8cd-b7f6-44b9-905c-532a17bc4a7d None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.969299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 468/937] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:40.975758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:40.976049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:40.976330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:40.996592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:40.998189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:41.014653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:41.073856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 470/938] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:41.096244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001f8cd-b7f6-44b9-905c-532a17bc4a7d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.210723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001f8cd-b7f6-44b9-905c-532a17bc4a7d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.246936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:41.247577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.247856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.248125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.259608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:41.264518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:41.265464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:41.266299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 469/939] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:41.276864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.390520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2102f3f-3afe-4ad0-a2e8-d8a9346978dc None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.429136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:41.429756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.430037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.430311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.431022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:41.431322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:41.432736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:41.446971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 471/940] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:41.448360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.453782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:41.562171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69961302-4a55-441f-8463-ab3610d7338a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.598065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:41.598703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.598977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.599248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.610794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:41.612945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7c4aa564-b1e5-4af3-b841-daa26ec09a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:41.628075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:41.629090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:41.629974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:41.629974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:41.629974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:41.629974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:41.629974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 470/941] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:41.635244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.651725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:41.674388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5be-99ef-4b80-984c-ff54ca6372e6 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.675596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5be-99ef-4b80-984c-ff54ca6372e6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.676078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111c5be-99ef-4b80-984c-ff54ca6372e6 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.677953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 471/942] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:41.682983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.683272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.684822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.713752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:41.715801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:41.759718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7429ad6-d6df-4419-bab1-17dad1441b87 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.788425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:41.797584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:41.798345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:41.798714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:41.799099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:41.800437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:41.800890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:41.802622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:41.815800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 472/943] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:41.824001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 472/944] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:41.833249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.835666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:41.936696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:41.954272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed99ec2-6b1d-4abf-9bfe-f4848f280e6a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.955402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1114f6-ff7f-4041-8fb0-9ae9fbd2ec15 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:41.999721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.000520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.000803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.001075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.003841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.005303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.005692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.006066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.006821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.008603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.009108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.010553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:42.020229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:42.021447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:42.022269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:42.027221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:42.029193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:42.030743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:42.031929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 473/946] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:42.031929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.032179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 473/946] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:42.032179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.092686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.159788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4b9cd-b245-45ac-9972-1fc08ddb8204 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.176967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5573dd-f1e4-4e43-a48d-e6dcee9eec43 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.197842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.198492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.198895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.199321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.200536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:42.200846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.202356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.220374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.221735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.222088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.222442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.223475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:42.223850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.225345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.228434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 474/947] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:42.230310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.231425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.239663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 474/948] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:42.242340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.354067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b7e33-8505-4aa6-9fc9-b93db4a28faa None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.357342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.358310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4a731-2e0f-4e35-ba1b-17436253ea84 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.395917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.396693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.396794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.396976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.397254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.397611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.397999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.398301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.398983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:42.399289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.400680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb599da-db9e-448d-955d-32797e6ae6c8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.405818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 475/949] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:42.406840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3856e-c407-41e7-a8dd-fd57a863652c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.407129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3856e-c407-41e7-a8dd-fd57a863652c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.407399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb3856e-c407-41e7-a8dd-fd57a863652c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.411319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.413363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 476/950] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:42.413880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.414780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.415073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.415351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:42.435210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:42.436169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:42.437092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:42.437092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:42.437092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:42.437092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:42.437321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:42.437321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:42.445824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 475/951] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:42.449478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.520646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.531928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 477/952] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:42.561793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.573769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f77dc-06ad-430a-88e8-9b6334c27a26 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.621379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.622481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.622885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.623286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.623986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:42.624565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.626331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.646322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 476/953] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:42.653400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.673403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.684533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705f400-ccf5-43c5-b2e9-77dd65fdb7bb None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.729546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.730378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.731020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.731297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.732208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.733673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:42.747516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:42.748998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:42.750125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:42.756359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 478/954] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:42.756359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.787510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef5e6-93c8-402d-ac3b-776120618b2b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.819623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:42.837657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.838305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.839108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.839391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.849171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.851033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.865658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c9dd-f621-4883-98cb-0b86dc24f92f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:42.866936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:42.868128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:42.869836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:42.874045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 477/955] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:42.875801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.914006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:42.914882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:42.915157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:42.915429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:42.916162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:42.916492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:42.919116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:42.928586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 479/956] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:42.932264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:42.979538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:43.012700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c6e69-08ee-409d-8d92-653ca1007b7d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.039998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957bd0de-fe76-4caf-b3a7-d02e3b8edc4d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.058186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.058909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.059245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.059588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.060431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:43.060738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.062201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295c375-cfcc-482b-97d7-a018f5a330d6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.066964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:43.075669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 478/957] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:43.077117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b92e-7c7f-43e6-993c-ac8538bba5bc None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.077456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b92e-7c7f-43e6-993c-ac8538bba5bc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.077740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570b92e-7c7f-43e6-993c-ac8538bba5bc None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.079172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 479/958] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:43.083724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570b92e-7c7f-43e6-993c-ac8538bba5bc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:43.085045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.085899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.086404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.086685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.088204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:43.088204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.089100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327444f-7362-460a-8b5c-1bcaacbc14c2 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.099406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 480/959] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:43.099406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.099406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.099406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.115891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:43.117051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:43.191639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 481/960] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:43.192933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:43.210772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570b92e-7c7f-43e6-993c-ac8538bba5bc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.250495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.251165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.251446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.251727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/1fb00a6bca4f48468574ff17083e1fbf/roles/c86c3ffc0dd24d849ebd736b1f1af775` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.254239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=1fb00a6bca4f48468574ff17083e1fbf, role_id=c86c3ffc0dd24d849ebd736b1f1af775)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.255670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.259132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:43.271630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 480/961] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:43 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/1fb00a6bca4f48468574ff17083e1fbf/roles/c86c3ffc0dd24d849ebd736b1f1af775 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:43.272558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:43.327846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd081b-1f68-4261-84aa-b8e7b4a8f834 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.348838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:43.370447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.371337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.371731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.372317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.373158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.374922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.400313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 482/962] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:43 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:43.420176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a1939-1fed-4148-b235-ac0d0288a550 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.470584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.471385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.472373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.472686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.473381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.474922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.477880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING keystone.server.flask.common [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 20:42:43.491905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.sql.core [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 20:42:43.491905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 20:42:43.491905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [parameters: {'user_id': '5af72c7a82e44346bc025fa1efb32637', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 20:42:43.491905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=86872) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result.read() Aug 04 20:42:43.498795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:42:43.499761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application yield resource Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application next(self.gen) Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise self.value Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application session.commit() Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 20:42:43.500806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.session.flush() Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application compat.raise_( Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 20:42:43.501693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 20:42:43.502648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application conn.query(q) Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application result.read() Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 20:42:43.503509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [parameters: {'user_id': '5af72c7a82e44346bc025fa1efb32637', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:43.505661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 20:42:43.507570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:43.512720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 481/963] 10.222.0.52 () {60 vars in 1131 bytes} [Thu Aug 4 20:42:43 2022] POST /identity/v3/users => generated 160 bytes in 228 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:43.513610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:43.630342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325aaf1-6649-48ea-8d21-0475501d8f68 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.669501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.670501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.671007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.671588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.681183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.682937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-59fae8c6-f161-42d3-8aba-dca2b7f2ab66 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:43.695670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:43.697161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 20:42:43.698099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:43.700696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 483/964] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:43.706963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:43.769693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:43.821415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b2831-a7df-4d91-9d57-2f68d795f616 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:43.858191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:43.859214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:43.859729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:43.860205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:43.861232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:43.861775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:43.863683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e518a-b223-42d3-bc9f-33f7fb5f01e7 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:43.904572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 482/965] 10.222.0.52 () {56 vars in 1101 bytes} [Thu Aug 4 20:42:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:44.495475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc47bb-6994-4c7a-be9e-b2e8f3541ce4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:44.495933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc47bb-6994-4c7a-be9e-b2e8f3541ce4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:44.496038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dc47bb-6994-4c7a-be9e-b2e8f3541ce4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:44.497635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 484/966] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:44.503859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d580-94ae-4b18-b5b1-1bfd2c3432d9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:44.504301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d580-94ae-4b18-b5b1-1bfd2c3432d9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:44.504623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d580-94ae-4b18-b5b1-1bfd2c3432d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:44.524145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b261d580-94ae-4b18-b5b1-1bfd2c3432d9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:44.526694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b261d580-94ae-4b18-b5b1-1bfd2c3432d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:44.604389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 483/967] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:44.623512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc47bb-6994-4c7a-be9e-b2e8f3541ce4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:44.656238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:44.732704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dc47bb-6994-4c7a-be9e-b2e8f3541ce4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:44.766579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:44.767200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:44.767485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:44.767761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:44.768830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:44.770206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c366bf-6db2-4ad8-b8fe-a7a85c014789 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:44.786767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 485/968] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:44 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:45.415428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c1865-7571-40a3-bd58-cc7f629ae420 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.415974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c1865-7571-40a3-bd58-cc7f629ae420 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.415974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c1865-7571-40a3-bd58-cc7f629ae420 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.417495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 484/969] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:45.422527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.422815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.423086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.442628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:45.443802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:45.525051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93a84-d3c6-46d6-b9b1-a255595bf309 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.525363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93a84-d3c6-46d6-b9b1-a255595bf309 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.525604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee93a84-d3c6-46d6-b9b1-a255595bf309 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.527898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 485/970] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:45.529886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 486/971] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:45.532028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.532322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.532629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.551836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:45.553604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:45.554445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:45.574009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:45.596169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:45.632324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 486/972] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:45.654486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:45.660430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b0f4a-3dd2-4f11-8886-d2811bc58746 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:45.694802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:45.695365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.695642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.695912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.706760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:45.711723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:45.712840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:45.716922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 487/973] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:45.719370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:45.767864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b31cf-519e-42c7-baa2-9d8f79110f9e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:45.806069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:45.807014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.807467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.807903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.808057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:45.808726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:45.810119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:45.819748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:45.820818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:45.821635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 487/974] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:45.827058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:45.835305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341904-1d91-411c-802e-c37e2fdf10b8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:45.863330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:45.880387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:45.880387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.880387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.880387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.880801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:45.880801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:45.881376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:45.891250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 488/975] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:45.896807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:45.934161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b19280-f7df-4c0e-87b5-902192aa85a3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:45.968979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:45.969515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:45.969794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:45.970069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:45.970789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:45.971091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:45.973242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:45.982831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 488/976] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:45.984027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.008603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313d5d5-64d0-44ad-a99a-394be7b1a38c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.043138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.043775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.044300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.044300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.054789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.057575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.059419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:46.081602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:46.082673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:46.083613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:46.083613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:46.083613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:46.083613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:46.087715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 489/977] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:46.088496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.092676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.095964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9d7d8e-4198-48cf-be67-b611ce768fac None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.131996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.132721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.133069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.133405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.134141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:46.134506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.135924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.141943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 489/978] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:46.148120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.197835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b2dfa-0357-426b-a9ab-db2b6ea37b62 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.232738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.233271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.233557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.233825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.234525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:46.234812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.236174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.248632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 490/979] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:46.255071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.259966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f31e2-bd69-4ffa-9d98-59898a1e9d03 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.299132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.300266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.300578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.300949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.301709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] RBAC: Adding query filter params (interface=admin, service_id=c365a22dedc24813ba15d39dc185a5e6, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:46.302021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.303635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.308839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 490/980] 10.222.0.52 () {56 vars in 1249 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/endpoints?service_id=c365a22dedc24813ba15d39dc185a5e6&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:46.313735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.314574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.345651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.368793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f4765-40a6-434c-9b29-4aec053b271c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.406442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.407385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.407670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.408323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.418308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.420276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.423522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54294e-92a0-4f28-8084-2e25f04b341a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:46.436806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:46.437970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 20:42:46.439133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:46.447059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 491/981] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:46.448617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.467472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.468096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.468405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.468689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.469339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.470701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.484498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 491/982] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:42:46.587220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7456b3-e61c-4e47-a082-d13ff2411ced None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.602315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.626807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.627464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.627728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.628005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.628852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:46.629163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.630614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a63bc-32b1-42af-9b5c-e8051bf918ac None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.650385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 492/983] 10.222.0.52 () {56 vars in 1149 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:46.656387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.755000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.765076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265d0e6d-140f-43c8-b1f8-1b22f0056097 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:46.803185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:46.803185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.803185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.803185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.838330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da547c8a-4c24-4548-b3ec-42a49ca1b6a9 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.838864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da547c8a-4c24-4548-b3ec-42a49ca1b6a9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.838864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da547c8a-4c24-4548-b3ec-42a49ca1b6a9 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.840223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 493/984] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:46.845179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:46.845463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:46.845748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:46.865273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:46.866515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:46.880317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:46.881792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:46.941295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 494/985] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:46.963075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:46.981022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:46.994867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 492/986] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:42:47.004390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.067715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.071317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ede720-bbfb-4b9a-8af8-8d685a006854 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.109794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.110703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.110980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.111248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] PATH_INFO: `/v3/services/compute` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.111898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.113034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee63505-a6d7-4c9d-88c3-52ba3edb8f93 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.113488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:47.122897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:47.123936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:47.124799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 495/987] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:47.129996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.152673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.154238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.154788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.154897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.166746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.168201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:47.183904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:47.185048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:47.185936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:47.185936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:47.185936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:47.185936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:47.185936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 493/988] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:47.190680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.241183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d2713-874b-45c5-ab2d-f2e76d1bc543 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.242000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.280175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.280939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.281219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.281493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.282161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.282459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.284478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.289464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 496/989] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:47.295312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.309578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe79b3a-8452-49f3-bbdb-97847dde12ca None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.324295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.350791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.351410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.351698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.351976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.352686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.352993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.354426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.371162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 494/990] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:47.372302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.406935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fbf33-14a0-453e-8790-760fe6b002f6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.447165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.447787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.448188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.448501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.449179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] RBAC: Adding query filter params (type=compute) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.449485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.450866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.457243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 497/991] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:47.462801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.483753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7c0af-d5e8-4610-a776-7f5fdac7320e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.486233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.522826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.523541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.523892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.524161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.531621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.533383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:47.546382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:47.547309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:47.548181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 495/992] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:47.554820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.575209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.587004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2bcbb8-ae7b-4322-a879-5ca1762eb5eb None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.624725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.625363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.625680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.625924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.626698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] RBAC: Adding query filter params (interface=public, service_id=0a1c80d2393b40c7a3d160f907b00ec1, region_id=RegionOne) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.627009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.628796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.638992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 498/993] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/endpoints?service_id=0a1c80d2393b40c7a3d160f907b00ec1&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:47.640012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.669892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26699859-f022-4a23-b8ae-310520c01b65 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.708554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.709169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.709445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.709714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.710454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.710753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.712169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.730278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 496/994] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:47.731275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:47.755269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc42b5-1fb6-45fc-8646-a68a729b4b16 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.760046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:47.794241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.794815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.795093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.795385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.796241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.797434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0b277d-0018-4e04-8841-edd859ad98c8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.805289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 499/995] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:42:47.851816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbf49c-91eb-434f-b390-733ddd4da377 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:47.890012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:47.890644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:47.890958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:47.891236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:47.892039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] RBAC: Adding query filter params (role.id=c86c3ffc0dd24d849ebd736b1f1af775, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=1fb00a6bca4f48468574ff17083e1fbf) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:47.892377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:47.894373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22251a-3279-4bf6-99ae-d02ab1c2d766 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:47.907544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 497/996] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:47 2022] GET /identity/v3/role_assignments?user.id=1fb00a6bca4f48468574ff17083e1fbf&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=c86c3ffc0dd24d849ebd736b1f1af775 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:48.540070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbcf0-1ecd-4273-ac63-5f60a87ae664 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:48.540623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbcf0-1ecd-4273-ac63-5f60a87ae664 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:48.540623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9bbcf0-1ecd-4273-ac63-5f60a87ae664 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:48.542139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 500/997] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:48.547248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:48.547639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:48.547823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:48.567454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:48.568675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:48.643744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 498/998] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:48.666572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bbcf0-1ecd-4273-ac63-5f60a87ae664 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:48.694012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:48.775946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9bbcf0-1ecd-4273-ac63-5f60a87ae664 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:48.809632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:48.810273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:48.810554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:48.810827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] PATH_INFO: `/v3/services/object-store` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:48.811442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:48.813407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:48.822955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:48.823954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 20:42:48.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:48.828425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 501/999] 10.222.0.52 () {56 vars in 1126 bytes} [Thu Aug 4 20:42:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:48.830470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:48.849862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:48.933000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6426e5ec-2018-4190-b284-ebe6a742e40c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:48.971781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:48.972488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:48.972777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:48.973056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:48.973752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:48.974057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:48.975454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:48.984670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 499/1000] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:48.987467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:49.076097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:49.093940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f846a0-0c80-4bb2-a05c-45fa4d52aeea None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:49.131020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:49.131640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.131922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.132198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.132917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:49.133225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:49.134594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5ac825-077b-4f0d-9628-da7dc3da0b0c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:49.140651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 502/1001] 10.222.0.52 () {56 vars in 1122 bytes} [Thu Aug 4 20:42:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:49.145831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:49.228997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:49.248101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0513ed-a1e2-47d1-8e7d-c8f0cfb66e59 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:49.282135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:49.282754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.283312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.283586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.284328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:49.285708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdb458-fac2-4cd1-9e71-bd167badaa43 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:49.312382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 500/1002] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:49 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:49.888892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335296a1-956d-41b6-b2d8-31c00ac3de70 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.889477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335296a1-956d-41b6-b2d8-31c00ac3de70 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.889477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335296a1-956d-41b6-b2d8-31c00ac3de70 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.890836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 503/1003] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:49.896367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.896758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.896939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.915786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:49.916989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:49.971800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c6d0-1831-431c-8b65-bacb31724a33 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.972162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c6d0-1831-431c-8b65-bacb31724a33 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.972542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54c6d0-1831-431c-8b65-bacb31724a33 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.974111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 504/1004] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:49.978972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:49.979261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:49.979541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:49.994175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 501/1005] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:49.998475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:49.999749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:50.016174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.081341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 505/1006] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:50.092158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:50.100923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.128278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c7eab-13cf-4907-8106-1ae0bb0ed340 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.165023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.165591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.165873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.166145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] PATH_INFO: `/v3/services/compute` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.166769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.168141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:50.177018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:50.178252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:50.179263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 502/1007] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:50.184166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.185299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:50.210296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b6b61-a42a-4b4a-a4a3-595d530e2932 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.245241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055758-1610-4715-853e-555b001b7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.245906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055758-1610-4715-853e-555b001b7cbc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.246165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055758-1610-4715-853e-555b001b7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.246442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51055758-1610-4715-853e-555b001b7cbc None admin] PATH_INFO: `/v3/services/image` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.247060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055758-1610-4715-853e-555b001b7cbc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.248454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51055758-1610-4715-853e-555b001b7cbc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-51055758-1610-4715-853e-555b001b7cbc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:50.257456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:50.258449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:50.264412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 506/1008] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:50.265652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055758-1610-4715-853e-555b001b7cbc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.292966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c2d16-fd0b-4762-a345-0a91f138502c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.347883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:50.354395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.355020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.355312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.355577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.356284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:50.356623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.360822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.368022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 503/1009] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:50.373437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.381479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51055758-1610-4715-853e-555b001b7cbc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.424884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.425435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.426178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.426178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.426753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] RBAC: Adding query filter params (name=image) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:50.427057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.428851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.434358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 507/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:50.439757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.450155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:50.489524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05240aac-8930-460a-a269-a30857300cc1 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.528503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.529124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.529405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.529675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.530349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:50.530650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.532014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.542721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 504/1011] 10.222.0.52 () {56 vars in 1112 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:50.543563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:50.558337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51491026-475e-4937-a99f-8c2b9c9dfeac None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.593261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.593874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.594156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.594430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.595094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] RBAC: Adding query filter params (type=image) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:50.595473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.597566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6568d-fa63-47a7-8b92-52ad9d6bb523 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.612419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 508/1012] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:50.612672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:50.658409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce10132-0c8d-4670-8c66-06f5e17045a1 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:50.702655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:50.703204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:50.703489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:50.703765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:50.704521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:50.705965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1a1806-59c1-4f5e-9025-8576352de6e5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:50.736389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 505/1013] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:51.395647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed45d-c8e3-4aaa-96dd-ab84a40ca79e None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:51.396170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed45d-c8e3-4aaa-96dd-ab84a40ca79e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:51.396424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ed45d-c8e3-4aaa-96dd-ab84a40ca79e None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:51.397791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 509/1014] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:51.402986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:51.403401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:51.403568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:51.421748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:51.422934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:51.500383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 506/1015] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:51 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:51.525094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed45d-c8e3-4aaa-96dd-ab84a40ca79e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:51.602371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:51.636842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ed45d-c8e3-4aaa-96dd-ab84a40ca79e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:51.672591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:51.673151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:51.673428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:51.673697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:51.675525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:51.677028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e77e2-dab8-4231-ad64-d1669af8ad19 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:51.683571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 510/1016] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:51.684659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:51.731820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:51.795048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d2d6b-86e7-422b-b22a-caf58744f95a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:51.829959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:51.831087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:51.831439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:51.831711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:51.832415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:51.833833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624349c-4b1c-49df-a8dd-cf96f7593476 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:51.864400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 507/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:51 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:42:52.629659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763e95e-247b-425e-8327-169fb7abed16 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:52.630157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763e95e-247b-425e-8327-169fb7abed16 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:52.630267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763e95e-247b-425e-8327-169fb7abed16 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:52.631539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 511/1018] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:52.637483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f37c9-6ce3-4664-9c7d-ad48dc074055 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:52.637955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f37c9-6ce3-4664-9c7d-ad48dc074055 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:52.638029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f37c9-6ce3-4664-9c7d-ad48dc074055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:52.660269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f64f37c9-6ce3-4664-9c7d-ad48dc074055 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:52.661922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f64f37c9-6ce3-4664-9c7d-ad48dc074055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:52.738453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 508/1019] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:42:52.761917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763e95e-247b-425e-8327-169fb7abed16 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:52.851967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f06a3-5edc-4d55-b354-c611bda92555 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:52.852143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f06a3-5edc-4d55-b354-c611bda92555 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:52.852451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f06a3-5edc-4d55-b354-c611bda92555 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:52.857558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 509/1020] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:52.858962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:52.859312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:52.859658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:52.878394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:52.879628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:52.885553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763e95e-247b-425e-8327-169fb7abed16 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:52.906273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:52.936573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:52.938540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:52.939429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:52.939705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:52.940970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:52.942344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:52.958948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:52.976589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 512/1021] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:42:52 2022] POST /identity/v3/services => generated 242 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:52.979895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 510/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:53.004205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.128802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30129206-2879-4d84-9889-2f0b80ebf226 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:53.168285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:53.168967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.169256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.169555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.181059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:53.186606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:53.187744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:53.189019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 513/1023] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:53.194278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.327073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0b58-481c-4eb4-8ece-c93358c43900 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:53.372950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:53.373576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.373854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.374124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.374890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:53.376077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:53.376685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:53.392119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 511/1024] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:53.392295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.454080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:53.499219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d2ab8-f818-4530-8f3d-0d13badb8dcf None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:53.538818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:53.539457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.539733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.540003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.552506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:53.553966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:53.569741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:53.570683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:53.576170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:53.576170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:53.576170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:53.576170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:53.576170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 514/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:53.576895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.593942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:53.684100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8262ed-79b8-4434-8848-d505a1004cee None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:53.719103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:53.719725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.720004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.720277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.721021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:53.721374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:53.722811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:53.740104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 512/1026] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:53.741391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.850998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f04822-cd9c-460a-9f5b-726cb8bc0b13 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:53.862136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:53.888523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:53.889166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.889449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.889724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.897678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:53.899116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ffb980b0-9756-44ba-852b-1abc8560f7d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:53.910574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:53.911480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:53.912463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:53.913729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 515/1027] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:53.917381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:53.972527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c4a3-9f38-4814-b526-b8dc5cb2dcbf None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.972826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c4a3-9f38-4814-b526-b8dc5cb2dcbf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.973004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a284c4a3-9f38-4814-b526-b8dc5cb2dcbf None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.974357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 516/1028] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:53.979792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:53.980520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:53.980846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:53.993373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.010564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:54.011762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:54.026510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6baa6e-cad0-407c-a883-68fe6eb91965 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.062875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.063501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.063789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.064068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.065561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:54.065890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.067339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.087661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 513/1029] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:54.093196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.107854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 517/1030] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:54.128819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.213774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352f80a-911e-43c9-84fb-2a34079d84ff None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.233530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.237763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77970538-ec27-43c9-84c5-e02c9691b878 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.245672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.253251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.253503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.253787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.254063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.276407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.277036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.277312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.277586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.278751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.280444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.287125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 518/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:54.287759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.331147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.332620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.399534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8a9b8-fafc-4c2a-8bc2-a2b9c7cfe491 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.437485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.438105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.438389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.444108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.444251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 514/1032] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:42:54.444333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.444333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.445893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.464823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 519/1033] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:42:54 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:42:54.489269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.579842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93fbb2-5a47-4b04-a38a-3dc8d0b0ce21 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.618566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354627-02c7-431f-9984-722e0387965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.619216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354627-02c7-431f-9984-722e0387965e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.619498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354627-02c7-431f-9984-722e0387965e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.619775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e354627-02c7-431f-9984-722e0387965e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.631211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e354627-02c7-431f-9984-722e0387965e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.632701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e354627-02c7-431f-9984-722e0387965e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0e354627-02c7-431f-9984-722e0387965e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:54.648590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:54.649656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:54.650661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:54.650661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:54.650661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:54.650661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:54.650661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 515/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:54.655562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.683539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.763348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f270ba8-7b95-4420-a15e-6f64ecf72d55 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.797265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.797887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.798169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.798442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.799684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:54.800005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.801423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.819009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 520/1035] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:54.820678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354627-02c7-431f-9984-722e0387965e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:54.907926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:54.933576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e354627-02c7-431f-9984-722e0387965e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:54.968587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:54.969226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:54.969504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:54.969776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:54.978356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:54.979847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-96209343-404b-4c9d-8b94-28f7b5b5de24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:54.992140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:54.993185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:54.994084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:54.996477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 516/1036] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:54.999468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.059865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22645b-a4b6-493b-9d77-39c9b6f921da None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.060201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22645b-a4b6-493b-9d77-39c9b6f921da None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.060469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22645b-a4b6-493b-9d77-39c9b6f921da None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.061841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 517/1037] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:55.066614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.066906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.067188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.090565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:55.091769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:55.104829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:55.127365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ecc86-051d-43b8-aac0-178f22a83dcd None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.167000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.167613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.167892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.176409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.176704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 518/1038] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:55.176816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:55.176816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.176816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.184093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 521/1039] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:55.190374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.193507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.268933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:55.300863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485cdcd-c1d9-4d31-b587-19d0d59d8eae None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.302946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e68038-eb5d-4eed-acd9-51ca81f03c66 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.337635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.338514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.338793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.339128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.339427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.339708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.339882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.339987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] PATH_INFO: `/v3/services/image` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.340614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.341152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] RBAC: Adding query filter params (role.id=c86c3ffc0dd24d849ebd736b1f1af775, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:55.341625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.341968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.343431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.350030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 519/1040] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=c86c3ffc0dd24d849ebd736b1f1af775 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:55.352153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:55.353208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:55.354066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:55.358497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 522/1041] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:55.359773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.481480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e2f81-ed7d-4011-981a-3bed23a04842 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.526331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.526859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.527140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.527412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.528093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] RBAC: Adding query filter params (name=image) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:55.528434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.529808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.532580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:55.538176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 520/1042] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:55.543384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.558254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:55.659805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c18f2e-c1ce-484a-9c34-cba2601b7696 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.698078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.698702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.698983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.699259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.699935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] RBAC: Adding query filter params (type=image) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:55.700240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.701638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.712035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 523/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:55.713188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.817421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfeed77-bfac-4c8c-92f5-1c77de6d5d48 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:55.853491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:55.854136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:55.854383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:55.854657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:55.855413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] RBAC: Adding query filter params (interface=public, service_id=8b6304e1966d4ee29b07d07a230fa458, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:55.855728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:55.857126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672456a8-b03c-4c73-9ef8-005233eaa728 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:55.868479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 521/1044] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/endpoints?service_id=8b6304e1966d4ee29b07d07a230fa458&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:55.868804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:55.945053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:55.977098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def52c82-2f9e-4dba-a5d3-d4ef761d33ee None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:56.012827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:56.013447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:56.013731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:56.014006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:56.014655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:56.016025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672eb05-1340-4a84-910e-d52c090546be None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:56.028575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 524/1045] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:42:55 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:42:56.573822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fb616-41b0-4651-a698-ffb72aa76c59 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:56.574157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fb616-41b0-4651-a698-ffb72aa76c59 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:56.574507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fb616-41b0-4651-a698-ffb72aa76c59 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:56.575995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 522/1046] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:42:56.581616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:56.581903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:56.582178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:56.600573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:56.601765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:56.670634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 525/1047] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:56.694272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fb616-41b0-4651-a698-ffb72aa76c59 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:56.759580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:56.803541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fb616-41b0-4651-a698-ffb72aa76c59 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:56.838563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:56.839207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:56.839488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:56.839760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:56.850344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:56.855193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:56.856573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:42:56.857461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:56.860661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 523/1048] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:42:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:56.863033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:56.930311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:56.966406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ef7fd-321f-4eeb-bb42-604cd020fdaa None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.000702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.001318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.001601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.001877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.003136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:57.003439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.004993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.017911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 526/1049] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:42:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:57.019393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.128484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac29eb-a09e-4848-b6c3-0d658c2ccb03 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.153680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:57.167406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.168057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.168361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.168642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] PATH_INFO: `/v3/users/fc10b64fd20044f191c5dfd510fff5bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.183264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] RBAC: Authorizing `identity:get_user(user_id=fc10b64fd20044f191c5dfd510fff5bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.184745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed7ac54-16a8-4924-a8fc-7a1615085a35 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.191741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 524/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/users/fc10b64fd20044f191c5dfd510fff5bb => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:57.192950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.294072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037101a-add6-4665-99c9-efafa0a9c005 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.320861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:57.329826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.330639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.331044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.331360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.383052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f688a87-d6ed-47e4-97c8-00352c2fa085 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.383248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f688a87-d6ed-47e4-97c8-00352c2fa085 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.383518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f688a87-d6ed-47e4-97c8-00352c2fa085 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.384907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 525/1051] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:57.389919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.390278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.390626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.405365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.406874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.408226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:57.411097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:57.481566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 526/1052] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:57.504156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.520369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 527/1053] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 20:42:57.527844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.558622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:57.619115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d0952-d81f-4f05-bde5-4f970726282a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.632853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:57.643851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c07a8-78e3-47ba-9b21-3d4db06dba03 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.665431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.666380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.666668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.666948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.678862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:42:57.682016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.682656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.682933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.683204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:42:57.683903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:57.684828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:42:57.685677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:57.686809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.686809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.692809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 528/1054] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:57.692968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 527/1055] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:57.693062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.694411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.789724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:57.796420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eb66b-8a85-411f-aa5e-0838e6ffbadc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.803987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905780b7-e3b0-4e5a-82ff-5192b12a2a2a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:57.839633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.840374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.841081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.841081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.841733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:57.842071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.843405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.844194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:57.845004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:57.845459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:57.845912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:57.846852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=c820024552894fcba4ec9db1816b1ae1, user.id=fc10b64fd20044f191c5dfd510fff5bb) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:57.847313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:57.848967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:57.851876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 529/1056] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:57.857705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:57.860051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 528/1057] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/role_assignments?user.id=fc10b64fd20044f191c5dfd510fff5bb&scope.project.id=c820024552894fcba4ec9db1816b1ae1&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:57.968251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74bee-24d9-401a-9212-d5d30a3e5641 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.005322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.005972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.006248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.006518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.019187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.020736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-5c8ea785-b758-4e25-a0b6-4681a01c073d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:58.035982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:58.036963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:58.038470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:58.038470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:58.038470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:58.038470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:58.038470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 530/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:58.043035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.081741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.088076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e027b8-af85-4554-acf3-2deb89ccbafc None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.088429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e027b8-af85-4554-acf3-2deb89ccbafc None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.088723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e027b8-af85-4554-acf3-2deb89ccbafc None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.090605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 531/1059] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:58.095454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.095743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.096857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.118527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:58.119799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:58.125076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.156432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa9c72-59d6-4782-b1b3-ea47194a0a6c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.195084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.195701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.195981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.196255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.196990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:58.197559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.197666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 532/1060] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:42:58.198830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.212787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 529/1061] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:58.222173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.222319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.324914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.332655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5522d-77a5-4744-b46a-1a48f755baf6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.334748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4471bdf-27ba-48fe-a23e-bf80ee92ab5c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.372318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.372994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.373281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.373560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] PATH_INFO: `/v3/services/image` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.374290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.374415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.375037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.375319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.375591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.375745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.384744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.386636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:58.388592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:58.389567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:58.390400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 530/1062] 10.222.0.52 () {56 vars in 1105 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:58.392689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.396990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:42:58.407208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:58.408180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:58.414798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 533/1063] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:58.415853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.504861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556bcfe-7b67-4f88-920e-3573a73d6270 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.526189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a760a-160f-4811-8036-325fc746ca73 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.551247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.551752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.552118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.552679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.553356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] RBAC: Adding query filter params (name=image) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:58.553655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.555047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.560481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 531/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:58.566748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.568333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.568989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.569273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.569550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.570351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:58.570662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.572089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.596658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 534/1065] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:58.596859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.598359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.679058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.692362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f24492-d8f7-4d17-9f35-ef4039c79ecb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.731150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b620db6-2377-4f83-8ada-2634880474fe None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.731690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.732308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.732641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.732883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.733551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] RBAC: Adding query filter params (type=image) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:58.733850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.735212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.741151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 532/1066] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:42:58.746701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.770939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.771581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.771856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.772124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.782184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.784136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:42:58.798364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:58.800798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:58.801917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:42:58.801917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:42:58.801917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:42:58.801917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:42:58.801917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 535/1067] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:42:58.806331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:58.850561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.865744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2aa157-a63e-406d-89a0-f226101b02f9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.910628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.911626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.912120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.912527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.913595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.916017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:58.926268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f51eb1-b553-4acc-a87f-29b4a455272b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:58.942541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:58.943650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 533/1068] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:42:58 2022] POST /identity/v3/endpoints => generated 327 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:42:58.988318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:58.989065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:58.989292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:58.989576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:58.990341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:58.990654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:58.992273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:59.014098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 536/1069] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:59.016016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:59.152613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815e2d9b-0768-444d-b006-f0f01bcbd6e2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:59.191282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:59.191931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:59.192217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:59.192529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:59.199301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:59.201358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:42:59.213416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:42:59.214469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:42:59.215673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:42:59.216852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 534/1070] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:42:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:42:59.223363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:59.327826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4ab5-ca22-4270-87e3-a810c0abf721 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:59.364946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:59.365732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:59.366111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:59.366498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:59.367319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:42:59.367724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:59.369293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ac93c-6259-48f6-a871-13e5c45d2468 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:59.386592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 537/1071] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:42:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:42:59.388269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:42:59.394555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:59.500235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3e7d5-03d2-42f2-a943-9516e456ef92 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:42:59.543325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:42:59.544129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:59.544545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:59.544927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/c86c3ffc0dd24d849ebd736b1f1af775` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:59.547186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=02a04385570944ecbcaf96003f5c5753, role_id=c86c3ffc0dd24d849ebd736b1f1af775)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:42:59.548760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed04289-97d0-4b6e-a5c1-14d4237d68c9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:42:59.562700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 535/1072] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:42:59 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/02a04385570944ecbcaf96003f5c5753/roles/c86c3ffc0dd24d849ebd736b1f1af775 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:42:59.887427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80de2-04ae-46be-bacb-6a669b38854e None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:59.887843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80de2-04ae-46be-bacb-6a669b38854e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:59.888021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b80de2-04ae-46be-bacb-6a669b38854e None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:59.889509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 538/1073] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:42:59.895053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:42:59.895516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:42:59.895682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:42:59.916188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:42:59.917428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:42:59.985266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:42:59.996246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 536/1074] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:42:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:00.018740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80de2-04ae-46be-bacb-6a669b38854e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:00.047448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:00.126970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b80de2-04ae-46be-bacb-6a669b38854e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:00.164197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:00.164807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:00.165090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:00.165370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:00.176755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:00.182840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:00.183756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:00.185089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 539/1075] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:00.190930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:00.297980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ca9c4-7efc-40be-b195-0c28659cc28d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:00.333316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:00.334257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:00.334750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:00.335228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:00.336160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:00.336677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:00.338227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:00.347440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 537/1076] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:00.353078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:00.439754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:00.455912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06485c63-90a4-44ce-b84b-6e7a7da2a77c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:00.491449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:00.492296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:00.492687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:00.493087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] PATH_INFO: `/v3/users/fc10b64fd20044f191c5dfd510fff5bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:00.494356 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] RBAC: Authorizing `identity:get_user(user_id=fc10b64fd20044f191c5dfd510fff5bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:00.495884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:00.502255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 540/1077] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:00 2022] GET /identity/v3/users/fc10b64fd20044f191c5dfd510fff5bb => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:00.503626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:00.570994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:00.617718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb9c52-eed8-423e-96bf-ab0d421a1656 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:00.651947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:00.652863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:00.653251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:00.653623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:00.654894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:00.656463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa65d07-7e06-486e-8569-003e700b201a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:00.663807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 538/1078] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:00 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:00.665029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:00.772636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26233e-ff3e-4a99-a913-718e59513e18 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:00.807903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:00.808773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:00.809158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:00.809533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1/users/fc10b64fd20044f191c5dfd510fff5bb/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:00.811551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=c820024552894fcba4ec9db1816b1ae1, user_id=fc10b64fd20044f191c5dfd510fff5bb, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:00.813118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03ed81-28be-4f1f-909e-6b3520fb30c6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:00.832387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 541/1079] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:43:00 2022] PUT /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1/users/fc10b64fd20044f191c5dfd510fff5bb/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:43:01.052149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289957b-4fb9-4617-9d71-a8ead7c5cc89 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.052950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289957b-4fb9-4617-9d71-a8ead7c5cc89 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.052950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289957b-4fb9-4617-9d71-a8ead7c5cc89 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.054401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 539/1080] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:01.059441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.059854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.060024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.078642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:01.079838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:01.089801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:01.153526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 542/1081] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:01.176223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289957b-4fb9-4617-9d71-a8ead7c5cc89 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:01.281956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289957b-4fb9-4617-9d71-a8ead7c5cc89 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:01.306989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:01.319295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:01.319998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.320364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.320719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.388871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:01.390427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927cda4-ca96-41c2-a415-f7e8430f42d0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:01.495687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 540/1082] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:01.496961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:01.609643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fab594-33e1-4c2a-a023-7243d016ba1d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:01.609941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba74a9c-eba3-4182-8242-5052155a3922 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.610148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba74a9c-eba3-4182-8242-5052155a3922 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.610428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba74a9c-eba3-4182-8242-5052155a3922 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.612031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 541/1083] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:01.616950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.617238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.617520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.635312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:01.636537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:01.645714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:01.646353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.646639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.646910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.657797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:01.657941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:01.660124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:01.668778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:43:01.678461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:01.679372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:01.680230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:43:01.680230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:43:01.680230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:01.680230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:01.680230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 543/1084] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:01.685391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:01.715803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 542/1085] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:43:01.748219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:01.797250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1619a9-a0d6-4ca1-bc02-7ad0455a8d96 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:01.836017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:01.837045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.837329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.837602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.838288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:01.838592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:01.840183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:01.855376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79515365-155e-44a2-b3d4-e491b9dbd9b0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:01.856713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 544/1086] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:01.894042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:01.894839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:01.895222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:01.895667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:01.911282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:01.916355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:01.917513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:01.918580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 543/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Thu Aug 4 20:43:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:01.926302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:01.992942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.045943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e10b7-2b4e-427a-a1a7-6447c3a7c9a3 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.086798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.087420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.087687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.087959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.088717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:02.089020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.090389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.099897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 545/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:43:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:02.105495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:02.122509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.211921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e82cf0-90a4-418c-aede-72e035916da9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.245453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.246090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.246373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.246650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.256547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.257998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:43:02.273850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:02.274919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:02.275804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:43:02.275804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:43:02.275804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:02.275804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:02.275804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 544/1089] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:02.280832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:02.380511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ce2883-f60d-4266-82e3-b1503053f6bc None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.397478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.416174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.416812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.417087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.417355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.418034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:02.418337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.419688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.432163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 546/1090] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:02.437577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:02.503723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.546111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df06bb-97ce-4a5e-844c-bc57085bf0d4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.581738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.582430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.582714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.583015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.591258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.592756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:43:02.604766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:02.605865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:02.606840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 545/1091] 10.222.0.52 () {56 vars in 1093 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:02.612456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:02.710191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b197ed32-4bed-43aa-9d01-a450941e145e None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.745016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.745474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.745783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.746080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.746931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:02.747240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.748719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb48fed-e525-4383-98e8-078d53b7ce34 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.765713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 547/1092] 10.222.0.52 () {56 vars in 1133 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:02.771984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:02.857104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daeea3-e855-4f47-a68d-3597daa4e941 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.857502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daeea3-e855-4f47-a68d-3597daa4e941 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.857845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daeea3-e855-4f47-a68d-3597daa4e941 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.859386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 548/1093] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:43:02.865018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.865301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.865572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.871830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a7a76-b2c9-4649-a532-610a59e55bd2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:02.894427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:02.895631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:02.897775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.906851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:02.907463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:02.907744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:02.908015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:02.926285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:02.965973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 549/1094] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:43:02.979063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:02.980562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:02.989912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.078325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 546/1095] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 20:43:03.084476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.088607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e421b73-5cdd-460f-9bec-0a893ce65c36 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.110507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.123587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.124221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.124530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.124836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.135044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:03.139910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:03.140865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:03.141860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 550/1096] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:03.147824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.176548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.186251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baba693-452c-4e6e-a410-b1a30fb1bfe3 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.220777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.221483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.221895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.222173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.233462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.234909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.249423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bce7-b113-4b01-ba9f-3ed1467daff0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:43:03.249597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:03.250449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:03.251353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:43:03.251353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:43:03.251353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:43:03.251353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:03.251353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 547/1097] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:03.257053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.286089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.286702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.286985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.287258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.287964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:03.288273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.289663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.297661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 551/1098] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.303981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.336681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.358522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1be62-c466-43f2-86d0-155b20dd297b None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.393543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.398596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.398596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.398596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.398596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.398994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:03.398994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.398994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.405560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6507c1-7a8c-4e25-9378-993103e2b456 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.411673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 548/1099] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.417257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.444728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.445364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.445642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.445913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] PATH_INFO: `/v3/users/fc10b64fd20044f191c5dfd510fff5bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.447062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] RBAC: Authorizing `identity:get_user(user_id=fc10b64fd20044f191c5dfd510fff5bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.448559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.450154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 552/1100] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/users/fc10b64fd20044f191c5dfd510fff5bb => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.455706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.518054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4ac6e-d1e8-408b-b9d1-62a9ede2b3f9 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.553976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.554607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.554884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.555160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.556245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dc02f-c212-47c1-928f-4eb26a6d8444 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.561677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.563117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.564948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:43:03.575702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:03.576694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:03.577568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 549/1101] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:03.583174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.590954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.591503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.591784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.592052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.617074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.660566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.662142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.683225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0d1eb-50a2-463d-8468-25f532f0413a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.716822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.717437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.717719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.717999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.718740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:03.719044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.720501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.733339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 550/1102] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.739466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.773284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 553/1103] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:03.779348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:03.799434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.836663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:03.841721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c70706-e77f-4109-929d-8069f8231925 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.878020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.878631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.878979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.879238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.880031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] RBAC: Adding query filter params (role.id=c86c3ffc0dd24d849ebd736b1f1af775, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=02a04385570944ecbcaf96003f5c5753) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:03.880365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.881809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.883930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f8795-4997-43ab-953b-3d92d2d68ede None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:03.887733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 551/1104] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/role_assignments?user.id=02a04385570944ecbcaf96003f5c5753&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=c86c3ffc0dd24d849ebd736b1f1af775 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.920314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:03.920990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:03.921268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:03.921538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:03.922758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:03.924646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963d228-76ac-4371-9dad-d7db7a6128cd None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:03.926420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 554/1105] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:03.932900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:04.038103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32409907-13d9-4142-a99a-5b613a7463a6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:04.061017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:04.074018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:04.074629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:04.074908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:04.075181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:04.075976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=c820024552894fcba4ec9db1816b1ae1, user.id=fc10b64fd20044f191c5dfd510fff5bb) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:04.076277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:04.077732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1b8ee-f86a-4c17-ab65-0bdae71425c2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:04.084799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 552/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:03 2022] GET /identity/v3/role_assignments?user.id=fc10b64fd20044f191c5dfd510fff5bb&scope.project.id=c820024552894fcba4ec9db1816b1ae1&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:06.184278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db421-9c87-4b05-b08f-1450429be0f5 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:06.185439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db421-9c87-4b05-b08f-1450429be0f5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:06.185718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713db421-9c87-4b05-b08f-1450429be0f5 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:06.187075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 555/1107] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:06.193227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:06.193635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:06.193786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:06.211362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:06.212564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:06.276912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 553/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:06.302904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713db421-9c87-4b05-b08f-1450429be0f5 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:06.402156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713db421-9c87-4b05-b08f-1450429be0f5 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:06.432959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:06.438411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:06.439067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:06.439496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:06.439941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:06.441089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:06.442517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f95e1-a11c-4b2c-af38-efa4354d1763 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:06.444137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 556/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:06.450400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:06.549504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15985182-0466-44ec-b865-e06ea55b83f5 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:06.565512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:06.584206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:06.584845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:06.585128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:06.585402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:06.586131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:06.587552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed30890-fb5f-4bac-a225-4f11ce2db8a0 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:06.612405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 554/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:43:06 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:08.633511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71ba4d-4989-4470-bc1a-02e4427483a0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:08.634086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71ba4d-4989-4470-bc1a-02e4427483a0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:08.634086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c71ba4d-4989-4470-bc1a-02e4427483a0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:08.635377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 557/1111] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:08.640560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:08.641003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:08.641122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:08.659256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:08.660472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:08.727518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 555/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:08.751460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71ba4d-4989-4470-bc1a-02e4427483a0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:08.852567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c71ba4d-4989-4470-bc1a-02e4427483a0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:08.887726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:08.888417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:08.888703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:08.888978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] PATH_INFO: `/v3/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:08.890408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] RBAC: Authorizing `identity:get_role(role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:08.891776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:08.893212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 558/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:08 2022] GET /identity/v3/roles/2b817625cc644df9915eaf992419b0c5 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:08.899903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:09.004421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209571b1-c31a-4ee5-aec8-e13f581ec3ca None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:09.006269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:09.041219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:09.041853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:09.042136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:09.042405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] PATH_INFO: `/v3/users/95a63904dd89465da30e851934f2be90` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:09.056785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] RBAC: Authorizing `identity:get_user(user_id=95a63904dd89465da30e851934f2be90)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:09.058247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:09.060058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 556/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:08 2022] GET /identity/v3/users/95a63904dd89465da30e851934f2be90 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:09.067301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:09.149397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:09.195250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c94dd7a-0e65-40a8-8687-86f03835ec1a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:09.233538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:09.234146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:09.234422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:09.234692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:09.313167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:09.315134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:09.378036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:09.434477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 559/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 20:43:09.440582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:09.556125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d1aca-7148-4449-b5f4-caaa8a2c81d6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:09.596435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:09.597087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:09.597368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:09.597645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:09.599000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:09.600483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ec9dec-10e3-4ec5-87a0-a4165fdcc40d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:09.602197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 557/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:09 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:09.608444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:09.723192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503164-fcc2-4c48-8b89-867a60fb9dfe None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:09.762764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18378576-3633-4091-bffc-d786963fffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:09.763439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378576-3633-4091-bffc-d786963fffeb None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:09.763729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378576-3633-4091-bffc-d786963fffeb None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:09.764104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18378576-3633-4091-bffc-d786963fffeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:09.764935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18378576-3633-4091-bffc-d786963fffeb None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=c820024552894fcba4ec9db1816b1ae1, user.id=95a63904dd89465da30e851934f2be90) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:09.765246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18378576-3633-4091-bffc-d786963fffeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:09.766692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18378576-3633-4091-bffc-d786963fffeb None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:09.773064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 560/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:09 2022] GET /identity/v3/role_assignments?user.id=95a63904dd89465da30e851934f2be90&scope.project.id=c820024552894fcba4ec9db1816b1ae1&role.id=2b817625cc644df9915eaf992419b0c5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:11.805003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817941-4147-4f9a-a278-a0ab54acac78 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:11.805524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817941-4147-4f9a-a278-a0ab54acac78 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:11.805524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53817941-4147-4f9a-a278-a0ab54acac78 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:11.806836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 558/1118] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:11.812011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:11.812465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:11.812759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:11.831322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:11.832546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:11.849699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:11.897855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 561/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:11.920578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53817941-4147-4f9a-a278-a0ab54acac78 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:11.950925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:12.025420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53817941-4147-4f9a-a278-a0ab54acac78 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:12.060273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:12.060866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:12.061143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:12.061413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] PATH_INFO: `/v3/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:12.062816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] RBAC: Authorizing `identity:get_role(role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:12.064177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:12.065646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 559/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:11 2022] GET /identity/v3/roles/2b817625cc644df9915eaf992419b0c5 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:12.071535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:12.168763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1af86-6c7b-45e5-91bc-dfc6e3bf5c89 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:12.202599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:12.203223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:12.203502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:12.203773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] PATH_INFO: `/v3/users/95a63904dd89465da30e851934f2be90` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:12.205561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] RBAC: Authorizing `identity:get_user(user_id=95a63904dd89465da30e851934f2be90)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:12.206991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:12.208614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 562/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:12 2022] GET /identity/v3/users/95a63904dd89465da30e851934f2be90 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:12.214388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:12.314263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:12.320483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862610d-7988-4a29-b7e1-9aef1d286333 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:12.355100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:12.355734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:12.356019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:12.356295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:12.357475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:12.358905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7821e7-ddc7-4aa5-97ab-ab63940b8955 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:12.360596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 560/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:12 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:12.366669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:12.390352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:12.463844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93d0a1-068c-41ee-a338-f1a3f56d52c1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:12.496745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b919402-685c-4051-9541-e884643a868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:12.497394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919402-685c-4051-9541-e884643a868d None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:12.497669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919402-685c-4051-9541-e884643a868d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:12.497937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b919402-685c-4051-9541-e884643a868d None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1/users/95a63904dd89465da30e851934f2be90/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:12.500035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b919402-685c-4051-9541-e884643a868d None admin] RBAC: Authorizing `identity:create_grant(project_id=c820024552894fcba4ec9db1816b1ae1, user_id=95a63904dd89465da30e851934f2be90, role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:12.501525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b919402-685c-4051-9541-e884643a868d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:12.519654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 563/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:43:12 2022] PUT /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1/users/95a63904dd89465da30e851934f2be90/roles/2b817625cc644df9915eaf992419b0c5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:43:14.501943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeac5-fc89-482a-991e-37464c93c573 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:14.502555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeac5-fc89-482a-991e-37464c93c573 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:14.502555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26efeac5-fc89-482a-991e-37464c93c573 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:14.504002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 561/1124] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:14.509229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:14.509607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:14.509775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:14.527953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:14.529162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:14.593836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 564/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:14.617244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efeac5-fc89-482a-991e-37464c93c573 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:14.718878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26efeac5-fc89-482a-991e-37464c93c573 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:14.752033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:14.752621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:14.752907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:14.753185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] PATH_INFO: `/v3/roles/2b817625cc644df9915eaf992419b0c5` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:14.754607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] RBAC: Authorizing `identity:get_role(role_id=2b817625cc644df9915eaf992419b0c5)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:14.755975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:14.757340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 562/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:14 2022] GET /identity/v3/roles/2b817625cc644df9915eaf992419b0c5 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:14.763299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:14.777924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:14.861382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f50b67-f406-450b-8ffc-9deb72835811 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:14.894537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:14.895153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:14.895429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:14.895696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] PATH_INFO: `/v3/users/95a63904dd89465da30e851934f2be90` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:14.896864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] RBAC: Authorizing `identity:get_user(user_id=95a63904dd89465da30e851934f2be90)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:14.898279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:14.899917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 565/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:14 2022] GET /identity/v3/users/95a63904dd89465da30e851934f2be90 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:14.905696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:14.936747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:15.004521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2a40-bb0b-49f0-98da-59998cc12527 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:15.038190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:15.038805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:15.039088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:15.039363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:15.106714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:15.108155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:15.137624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:15.207143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 563/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:15.213095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:15.291805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:15.311026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8eeb2-fdda-435a-a4bf-0690b779c5a7 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:15.346014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:15.346649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:15.346930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:15.347204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] PATH_INFO: `/v3/projects/c820024552894fcba4ec9db1816b1ae1` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:15.348371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] RBAC: Authorizing `identity:get_project(project_id=c820024552894fcba4ec9db1816b1ae1)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:15.349786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d53c2-dfc6-48ad-b234-548cee61d38c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:15.351442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 566/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:15 2022] GET /identity/v3/projects/c820024552894fcba4ec9db1816b1ae1 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:15.357521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:15.457110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e832c-ccae-48af-b2ad-7fa674a3d6f4 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:15.485411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:15.491412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:15.492025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:15.492306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:15.492603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:15.493383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] RBAC: Adding query filter params (role.id=2b817625cc644df9915eaf992419b0c5, scope.project.id=c820024552894fcba4ec9db1816b1ae1, user.id=95a63904dd89465da30e851934f2be90) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:15.493691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:15.495117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b075cce-24df-4961-a4ce-eb6486efd520 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:15.500886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 564/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:15 2022] GET /identity/v3/role_assignments?user.id=95a63904dd89465da30e851934f2be90&scope.project.id=c820024552894fcba4ec9db1816b1ae1&role.id=2b817625cc644df9915eaf992419b0c5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:17.539531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cead2-c589-4d34-a348-fa733adad139 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:17.539985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cead2-c589-4d34-a348-fa733adad139 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:17.540076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cead2-c589-4d34-a348-fa733adad139 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:17.541468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 567/1131] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:17.546725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:17.547102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:17.547275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:17.566275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:17.567433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:17.632848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 565/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:17.655662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cead2-c589-4d34-a348-fa733adad139 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:17.758158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cead2-c589-4d34-a348-fa733adad139 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:17.793488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:17.794055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:17.794355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:17.794628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:17.795768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:17.797239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06a6468-fde4-4b86-a94c-64d4234a4b56 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:17.799560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 568/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:17.806279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:17.902875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa7aba-bd46-4aa1-99f2-e62264401abc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:17.928288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:17.937267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:17.937874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:17.938152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:17.938419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:17.939721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:17.941175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f1a2-ea08-4b54-a447-021eadae936a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:17.967372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 566/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:43:17 2022] POST /identity/v3/projects => generated 311 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:20.004942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32e5f-f6fa-4827-881b-e1e167c3af7c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:20.005478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32e5f-f6fa-4827-881b-e1e167c3af7c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:20.005478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af32e5f-f6fa-4827-881b-e1e167c3af7c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:20.006751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 569/1135] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:20.011914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:20.012301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:20.012499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:20.030741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:20.031919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:20.098014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 567/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:43:20.122569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32e5f-f6fa-4827-881b-e1e167c3af7c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:20.140753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:20.226640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af32e5f-f6fa-4827-881b-e1e167c3af7c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:20.260192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:20.260787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:20.261061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:20.261336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] PATH_INFO: `/v3/domains/default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:20.262489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:20.263914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c504e-d650-4dc4-9ddc-3b6c9a2f63db None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:20.265694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 570/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:43:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:20.271919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:20.370849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5f98d-855c-44a5-8cb4-2debda264b2e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:20.379840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:20.405365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:20.405978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:20.406264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:20.406542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:20.407973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:20.409447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7192f0-73f1-414d-ac56-0277799ea193 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:20.431875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 568/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:43:20 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:22.487768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28023-70b9-4847-ab36-01dc749d7503 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:22.488306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28023-70b9-4847-ab36-01dc749d7503 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:22.488306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee28023-70b9-4847-ab36-01dc749d7503 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:22.489672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 571/1139] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:22.495264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:22.495706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:22.495819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:22.513837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:22.515070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:22.584945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 569/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:22.609597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28023-70b9-4847-ab36-01dc749d7503 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:22.696575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:22.713360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee28023-70b9-4847-ab36-01dc749d7503 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:22.747686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:22.748245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:22.748556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:22.748828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:22.760241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:22.765088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:22.766259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:22.767905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 572/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:22.773265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:22.864139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:22.871026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c5f39f-c8f1-4e21-9780-cba67cb406fc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:22.905911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:22.906525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:22.906810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:22.907088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:22.907924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:22.908366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:22.909741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:22.918312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 570/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:22.924754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:23.023601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1060c-5bee-4944-b82e-c69b1977fd2d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:23.057937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:23.058565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:23.058851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:23.059131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] PATH_INFO: `/v3/users/a2244fae090441669e1add9553e88308` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:23.073430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] RBAC: Authorizing `identity:get_user(user_id=a2244fae090441669e1add9553e88308)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:23.074868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:23.076508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 573/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:22 2022] GET /identity/v3/users/a2244fae090441669e1add9553e88308 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:23.082808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:23.184627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981db24-0424-4015-bfc8-cf0bf4290434 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:23.218647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:23.219260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:23.219541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:23.220939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:23.247229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:23.291849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:23.293251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:23.391542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 571/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:23.398063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:23.403099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:23.504288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c87af9-860b-47c1-90d9-594c7c283d10 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:23.539724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:23.540383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:23.540665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:23.540939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] PATH_INFO: `/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:23.542142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] RBAC: Authorizing `identity:get_project(project_id=5598d2b11d3e40da8e4eb7ee422ab75b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:23.543511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4768b849-2d4b-4ed6-8236-da2a2ee197e9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:23.545203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 574/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:23 2022] GET /identity/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:23.551352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:23.608353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:23.653728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93a35f-e924-4cef-a988-eca059c4e9d8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:23.687485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:23.688100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:23.688411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:23.689463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:23.690342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=5598d2b11d3e40da8e4eb7ee422ab75b, user.id=a2244fae090441669e1add9553e88308) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:23.690653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:23.692090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e891e0a5-fd2b-4d29-ad49-7e7c2bd6cec4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:23.697474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 572/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:23 2022] GET /identity/v3/role_assignments?user.id=a2244fae090441669e1add9553e88308&scope.project.id=5598d2b11d3e40da8e4eb7ee422ab75b&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:25.734234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25789-f093-4419-aa42-50c3c447e0e0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:25.734779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25789-f093-4419-aa42-50c3c447e0e0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:25.734886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb25789-f093-4419-aa42-50c3c447e0e0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:25.737192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 575/1147] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:25.744473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:25.744950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:25.745055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:25.762550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:25.763722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:25.831885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 573/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:43:25.854596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb25789-f093-4419-aa42-50c3c447e0e0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:25.924153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:25.957083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb25789-f093-4419-aa42-50c3c447e0e0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:25.997776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:25.998406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:25.998686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:25.998959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:26.009173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:26.013984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:26.015055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:26.016158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 576/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:26.022031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:26.074494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:26.123216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bacea3-76ee-44d2-9c87-6f3b70b4ea5c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:26.156302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:26.156944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:26.157303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:26.157584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:26.158296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:26.158604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:26.160031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:26.167860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 574/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:26.174355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:26.273119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66878814-8486-4919-9466-55c5432cf9f8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:26.308094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:26.315826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:26.316570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:26.316900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:26.317232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] PATH_INFO: `/v3/users/a2244fae090441669e1add9553e88308` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:26.318525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] RBAC: Authorizing `identity:get_user(user_id=a2244fae090441669e1add9553e88308)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:26.320068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:26.321773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 577/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:26 2022] GET /identity/v3/users/a2244fae090441669e1add9553e88308 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:43:26.327714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:26.428629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ab44d-cf79-4bb2-aed8-b9b84926c9f7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:26.441864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:26.463115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:26.463750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:26.464032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:26.464308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] PATH_INFO: `/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:26.465524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] RBAC: Authorizing `identity:get_project(project_id=5598d2b11d3e40da8e4eb7ee422ab75b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:26.466957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f060a0-ebff-4606-b1fb-03a73d8e9cae None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:26.468645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 575/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:26 2022] GET /identity/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:26.474740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:26.585605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab0752-c166-48fb-a597-3b43dc492002 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:26.624064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:26.624763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:26.625048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:26.625327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] PATH_INFO: `/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b/users/a2244fae090441669e1add9553e88308/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:26.627308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] RBAC: Authorizing `identity:create_grant(project_id=5598d2b11d3e40da8e4eb7ee422ab75b, user_id=a2244fae090441669e1add9553e88308, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:26.628776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cd48bc-a567-4837-bb50-5dd4dd574267 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:26.642739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 578/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:43:26 2022] PUT /identity/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b/users/a2244fae090441669e1add9553e88308/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:43:28.658992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970872-20b7-4b7a-84c1-67dd4b348a7b None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:28.659523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970872-20b7-4b7a-84c1-67dd4b348a7b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:28.659523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4970872-20b7-4b7a-84c1-67dd4b348a7b None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:28.660926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 576/1154] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:28.666184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:28.666577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:28.666733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:28.685320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:28.686506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:28.739828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:28.754425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 579/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:28.777843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4970872-20b7-4b7a-84c1-67dd4b348a7b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:28.883290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4970872-20b7-4b7a-84c1-67dd4b348a7b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:28.916917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:28.917549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:28.917838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:28.918114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:28.929051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:28.933920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:28.934978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:28.935818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 577/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:28.941571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:29.054659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c45e2-c5b6-4c3b-a8dd-decad7fc8e84 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:29.087901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:29.088530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:29.088807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:29.089072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:29.089773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:29.090069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:29.091429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:29.100331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 580/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:29.106393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:29.152606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:29.217375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3491612f-3d8c-42f1-926a-daea8517edb7 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:29.251867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:29.252539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:29.252815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:29.253083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] PATH_INFO: `/v3/users/a2244fae090441669e1add9553e88308` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:29.254241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] RBAC: Authorizing `identity:get_user(user_id=a2244fae090441669e1add9553e88308)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:29.255668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:29.257298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 578/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:29 2022] GET /identity/v3/users/a2244fae090441669e1add9553e88308 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:29.263186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:29.312880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:29.365740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165d4d8-69ad-4470-8855-bd8d75a5fe8d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:29.398918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:29.399527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:29.399808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:29.400081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:29.465777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:29.467234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:29.514721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:29.567311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 581/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:29.573235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:29.677609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb2bfa-b2ac-4b49-9e7c-ec749c8e5225 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:29.686424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:29.713056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:29.713695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:29.713977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:29.714255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] PATH_INFO: `/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:29.715389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] RBAC: Authorizing `identity:get_project(project_id=5598d2b11d3e40da8e4eb7ee422ab75b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:29.716859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7f1515-2b35-4b32-b8d9-c0c308170a4c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:29.718520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 579/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:29 2022] GET /identity/v3/projects/5598d2b11d3e40da8e4eb7ee422ab75b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:29.724764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:29.823114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55f171-6e6f-4211-90b7-0d569388a58f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:29.855088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:29.855687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:29.855963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:29.856234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:29.857038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=5598d2b11d3e40da8e4eb7ee422ab75b, user.id=a2244fae090441669e1add9553e88308) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:29.857538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:29.858849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a2e5c-2aa4-4b90-bdda-f68c653ba6b4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:29.865085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 582/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:29 2022] GET /identity/v3/role_assignments?user.id=a2244fae090441669e1add9553e88308&scope.project.id=5598d2b11d3e40da8e4eb7ee422ab75b&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:31.902131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311fb9-83f4-47d9-9b64-25dc4617194d None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:31.902639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311fb9-83f4-47d9-9b64-25dc4617194d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:31.902639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311fb9-83f4-47d9-9b64-25dc4617194d None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:31.903954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 580/1162] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:31.909820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:31.910213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:31.910367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:31.929341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:31.930532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:31.940297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:32.000035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 583/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:32.022415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311fb9-83f4-47d9-9b64-25dc4617194d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:32.124855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311fb9-83f4-47d9-9b64-25dc4617194d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:32.160116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:32.160684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:32.160969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:32.161247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:32.206858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:32.230588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:32.232046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:32.333284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 581/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:32.339384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:32.440641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7375b00-b1bd-4566-bdc8-503cd100d6cb None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:32.477463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:32.478104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:32.478382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:32.478654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:32.486157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:32.487809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-fe9e7749-ae34-4e2b-a27a-1d324590a867 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:32.499150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:32.504453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:32.505486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 584/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:43:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:43:32.507615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:32.589737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:32.613795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4261cd2-114e-47ec-a99d-4fece889214c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:32.648863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:32.649480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:32.649766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:32.650037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:32.650723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:32.651022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:32.652414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309c16d-2bcc-49cd-9237-e96bb4b6d820 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:32.660860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 582/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Thu Aug 4 20:43:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:34.673507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123381c7-12a3-4ecc-9580-0f5993d573d7 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:34.674073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123381c7-12a3-4ecc-9580-0f5993d573d7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:34.674357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123381c7-12a3-4ecc-9580-0f5993d573d7 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:34.675755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 585/1167] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:34.681509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508d69-8461-4f76-9b33-240e7987d220 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:34.681941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508d69-8461-4f76-9b33-240e7987d220 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:34.682163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508d69-8461-4f76-9b33-240e7987d220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:34.700668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ed508d69-8461-4f76-9b33-240e7987d220 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:34.701884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ed508d69-8461-4f76-9b33-240e7987d220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:34.771251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 583/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:34.794194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123381c7-12a3-4ecc-9580-0f5993d573d7 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:34.798499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:34.897500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123381c7-12a3-4ecc-9580-0f5993d573d7 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:34.931184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:34.931793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:34.932078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:34.932376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:34.933070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:34.934420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39239cfb-5940-4a97-9b18-a952ab7388fe None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:34.950472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 586/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Thu Aug 4 20:43:34 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:37.008582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d14a-2a0d-4cf0-9a58-e7ac3fcd2e4a None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:37.009246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d14a-2a0d-4cf0-9a58-e7ac3fcd2e4a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:37.009246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010d14a-2a0d-4cf0-9a58-e7ac3fcd2e4a None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:37.010366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 584/1170] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:37.015618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486865c-4850-4584-91be-f77f015dc650 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:37.016182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486865c-4850-4584-91be-f77f015dc650 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:37.016934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486865c-4850-4584-91be-f77f015dc650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:37.035770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9486865c-4850-4584-91be-f77f015dc650 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:37.037008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9486865c-4850-4584-91be-f77f015dc650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:37.101698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:37.112184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 587/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:43:37.136002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d14a-2a0d-4cf0-9a58-e7ac3fcd2e4a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:37.211982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:37.244523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010d14a-2a0d-4cf0-9a58-e7ac3fcd2e4a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:37.277938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:37.278576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:37.278859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:37.279129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] PATH_INFO: `/v3/domains/3dd043c0bae244018975dbb134912db3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:37.290103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3dd043c0bae244018975dbb134912db3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:37.291571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bf603-6769-4b5b-b30f-e77f6f328a79 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:37.293293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 585/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Aug 4 20:43:37 2022] GET /identity/v3/domains/3dd043c0bae244018975dbb134912db3 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:37.300197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486865c-4850-4584-91be-f77f015dc650 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:37.403823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486865c-4850-4584-91be-f77f015dc650 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:37.440746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:37.441331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:37.441610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:37.441880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:37.442558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:37.443969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e32c9e-2f78-4a0b-9bf2-1858461e6aa5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:37.470482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 588/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:43:37 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:39.512624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ce01-4f9c-4a2c-aae9-1c1c1dd63c42 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:39.513228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ce01-4f9c-4a2c-aae9-1c1c1dd63c42 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:39.513228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594ce01-4f9c-4a2c-aae9-1c1c1dd63c42 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:39.514548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 586/1174] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:39.521174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:39.521174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:39.522314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:39.540661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:39.542212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:39.588147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:39.616099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 589/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:39.641237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594ce01-4f9c-4a2c-aae9-1c1c1dd63c42 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:39.748695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594ce01-4f9c-4a2c-aae9-1c1c1dd63c42 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:39.775976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:39.782700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:39.783330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:39.783611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:39.783887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] PATH_INFO: `/v3/domains/3dd043c0bae244018975dbb134912db3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:39.785040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=3dd043c0bae244018975dbb134912db3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:39.786539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b91320-fd5b-453f-b53a-11ec99263e0c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:39.788114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 587/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Thu Aug 4 20:43:39 2022] GET /identity/v3/domains/3dd043c0bae244018975dbb134912db3 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:39.793922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:39.890218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312d267-f26d-4232-8d5a-b1455cc160d0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:39.923154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:39.923759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:39.924038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:39.924310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:39.925019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:39.926442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89850789-3957-4b3d-ba26-f5a5803f19ec None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:39.947205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 590/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:43:39 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:43:41.983229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f714df-71b5-47bc-a70b-b96ee4870b53 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:41.983746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f714df-71b5-47bc-a70b-b96ee4870b53 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:41.983746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f714df-71b5-47bc-a70b-b96ee4870b53 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:41.985304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 588/1178] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:41.990418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:41.990820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:41.990972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:42.010291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:42.011482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:42.037597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:42.080175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 591/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:42.103932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f714df-71b5-47bc-a70b-b96ee4870b53 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:42.208504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f714df-71b5-47bc-a70b-b96ee4870b53 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:42.242275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:42.242843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:42.243135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:42.243406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:42.254260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:42.259120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:42.260301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:42.261317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 589/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:42.266502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:42.362103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab70a8-efe4-4e24-8215-2d41e953081b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:42.395537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:42.396150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:42.396465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:42.396743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:42.397453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:42.397755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:42.399116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:42.407648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 592/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:42.413504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:42.474390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:42.516123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58aedbb-0b0b-4ec5-acd5-59ab6bf8942d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:42.550966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:42.551845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:42.552133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:42.552438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] PATH_INFO: `/v3/users/04c81938e184464e9fcef8930e87cb63` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:42.566143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] RBAC: Authorizing `identity:get_user(user_id=04c81938e184464e9fcef8930e87cb63)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:42.567589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:42.569257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 590/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:42 2022] GET /identity/v3/users/04c81938e184464e9fcef8930e87cb63 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:42.575164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:42.601596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:42.678327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc1099-f7a4-4c70-87e6-36caab968258 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:42.713349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:42.713953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:42.714235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:42.714514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:42.781265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:42.783423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:42.807388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:42.893268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 593/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:42.899260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:42.997590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb04d83-2aaf-4446-8611-34a722bc9cdd None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:43.013584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:43.032412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:43.032970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:43.033257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:43.033531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] PATH_INFO: `/v3/projects/728763d40d954b6c9921d2a08ad7bd88` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:43.034683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] RBAC: Authorizing `identity:get_project(project_id=728763d40d954b6c9921d2a08ad7bd88)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:43.036114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34966ad2-fc2c-4675-992c-4ee4eb2f26b9 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:43.037804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 591/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:42 2022] GET /identity/v3/projects/728763d40d954b6c9921d2a08ad7bd88 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:43.044282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:43.142766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57bc50d-11d9-4eee-8b11-a72a8da4b0e8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:43.167088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:43.179139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:43.179756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:43.180036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:43.180309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:43.181126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=728763d40d954b6c9921d2a08ad7bd88, user.id=04c81938e184464e9fcef8930e87cb63) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:43.181431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:43.182856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a338ab37-b5dc-4b41-a333-62847ecf8dc1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:43.188245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 594/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:43 2022] GET /identity/v3/role_assignments?user.id=04c81938e184464e9fcef8930e87cb63&scope.project.id=728763d40d954b6c9921d2a08ad7bd88&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:45.211463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813c69-dedd-4b68-98bf-c99a82f73849 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.212021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813c69-dedd-4b68-98bf-c99a82f73849 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.212021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813c69-dedd-4b68-98bf-c99a82f73849 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.213378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 592/1186] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:45.218721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.219657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.219917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.237840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:45.239026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:45.307003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 595/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:45.330134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813c69-dedd-4b68-98bf-c99a82f73849 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:45.435280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813c69-dedd-4b68-98bf-c99a82f73849 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:45.470754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:45.471318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.471600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.471877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.481842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:45.487213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:45.488316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:43:45.489381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 593/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:45.494498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:45.598389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cb824-6e0e-4efe-be8c-8390cf17aa4c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:45.616421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:45.634025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:45.634641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.634918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.635186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.635889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:45.636186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:45.637581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:45.647095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 596/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:45.653220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:45.700978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:45.756190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80976310-da53-4bd3-92df-d83a41a15bdb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:45.789594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:45.790227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.790511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.790788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] PATH_INFO: `/v3/users/04c81938e184464e9fcef8930e87cb63` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.792286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] RBAC: Authorizing `identity:get_user(user_id=04c81938e184464e9fcef8930e87cb63)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:45.793756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:45.795354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 594/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:45 2022] GET /identity/v3/users/04c81938e184464e9fcef8930e87cb63 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:45.802427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:45.901285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c18a9-3bec-4bc5-8c2c-e1c4740bdee9 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:45.934493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:45.935115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:45.935388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:45.935658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] PATH_INFO: `/v3/projects/728763d40d954b6c9921d2a08ad7bd88` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:45.936807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] RBAC: Authorizing `identity:get_project(project_id=728763d40d954b6c9921d2a08ad7bd88)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:45.938779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22bbd1-9eb7-45e5-a815-561621a4ba6c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:45.940438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 597/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:45 2022] GET /identity/v3/projects/728763d40d954b6c9921d2a08ad7bd88 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:45.946380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:46.042487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218b8cd-cc76-4190-a216-92a01b20b6ef None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:46.056193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:46.077170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:46.077827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:46.078223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:46.078509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] PATH_INFO: `/v3/projects/728763d40d954b6c9921d2a08ad7bd88/users/04c81938e184464e9fcef8930e87cb63/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:46.080585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] RBAC: Authorizing `identity:create_grant(project_id=728763d40d954b6c9921d2a08ad7bd88, user_id=04c81938e184464e9fcef8930e87cb63, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:46.082007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610fca7-0a96-45ae-9d40-959be1db03ca None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:46.094649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 595/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:43:45 2022] PUT /identity/v3/projects/728763d40d954b6c9921d2a08ad7bd88/users/04c81938e184464e9fcef8930e87cb63/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:43:48.096071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771174e-6e8c-4e1c-b1f1-9a1cca87c6a1 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.096653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771174e-6e8c-4e1c-b1f1-9a1cca87c6a1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.096653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771174e-6e8c-4e1c-b1f1-9a1cca87c6a1 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.098016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 598/1193] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:43:48.103305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.103704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.103854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.122802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:43:48.123999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:43:48.196029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 596/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:43:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:43:48.221397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771174e-6e8c-4e1c-b1f1-9a1cca87c6a1 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:48.273997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:48.327489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771174e-6e8c-4e1c-b1f1-9a1cca87c6a1 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:48.362711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:48.363260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.363540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.363810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.374199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:43:48.379665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:43:48.380800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:43:48.381837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 599/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:43:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:43:48.386986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:48.506518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460651b-3c6c-4f42-ade9-6a94faa3159d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:48.508178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:48.544832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:48.545447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.545724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.546004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.546716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:48.547111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:48.548807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:48.557671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 597/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:43:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:48.564858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:48.665599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7fd99e-7e9e-43ef-ac63-346760cdff26 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:48.674929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:48.700876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:48.701552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.701887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.702169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] PATH_INFO: `/v3/users/04c81938e184464e9fcef8930e87cb63` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.704713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] RBAC: Authorizing `identity:get_user(user_id=04c81938e184464e9fcef8930e87cb63)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:48.706169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:48.707846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 600/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Thu Aug 4 20:43:48 2022] GET /identity/v3/users/04c81938e184464e9fcef8930e87cb63 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:48.714374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:48.822071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58300e2e-5f01-4450-9622-99dfefbfda36 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:48.860589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:48.861202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:48.861486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:48.861769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:48.892684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:48.929826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:48.931195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:49.028277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 598/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:43:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:43:49.035174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:49.134539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16968c-c6cf-44dd-aa8f-b0fad46d519c None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:49.169210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:49.169846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:49.170125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:49.170416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] PATH_INFO: `/v3/projects/728763d40d954b6c9921d2a08ad7bd88` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:49.171537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] RBAC: Authorizing `identity:get_project(project_id=728763d40d954b6c9921d2a08ad7bd88)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:49.173583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b5b6b6-0c0e-44e9-bae7-43aedc128b88 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:49.175241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 601/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Thu Aug 4 20:43:49 2022] GET /identity/v3/projects/728763d40d954b6c9921d2a08ad7bd88 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:49.181743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:43:49.244661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:49.284211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad196d3-7cc9-41e6-9181-eba6d9c95472 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:43:49.320998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:43:49.321617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:43:49.321899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:43:49.322173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:43:49.322951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=728763d40d954b6c9921d2a08ad7bd88, user.id=04c81938e184464e9fcef8930e87cb63) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:43:49.323259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:43:49.324710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eafb27-f9da-48bc-9b2d-02d59a1b05a2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:43:49.331028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 599/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:43:49 2022] GET /identity/v3/role_assignments?user.id=04c81938e184464e9fcef8930e87cb63&scope.project.id=728763d40d954b6c9921d2a08ad7bd88&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:43:59.165024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=78 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:43:59.328432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=36 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:45.436600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f71133-6950-403c-8a68-a7ca112e083b None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:45.437229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f71133-6950-403c-8a68-a7ca112e083b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:45.437229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f71133-6950-403c-8a68-a7ca112e083b None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:45.438912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 602/1201] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:44:45.445263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:45.445811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:45.445811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:45.467453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:44:45.468744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:44:45.574074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 600/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:44:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:44:45.603415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f71133-6950-403c-8a68-a7ca112e083b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:45.726841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f71133-6950-403c-8a68-a7ca112e083b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:45.768028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:45.769013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:45.769476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:45.769958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:45.781931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:45.783588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:45.799916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:45.801460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:45.806078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:45.806078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:45.806078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:45.806078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:45.806078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 603/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:45.808172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:45.941126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc0563-2699-41ba-88f2-a587a6ea0e31 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:45.996879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:46.008351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:46.009548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:46.009990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:46.010284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:46.011191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:46.011504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:46.013116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4aef6-9ad8-4a80-978f-73d5a2063570 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:46.026955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 601/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:46.033306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:46.112775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:46.175223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463916e3-8212-4613-9db4-e4f56dcce5c6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:46.212917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:46.213685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:46.213975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:46.214253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:46.214982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:46.217719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f1dc9-bf47-4adc-af7b-22e75bee5716 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:46.244007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 604/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Thu Aug 4 20:44:46 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:44:48.448781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9b44-01d2-4af7-8945-b23e1844d941 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:48.449331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9b44-01d2-4af7-8945-b23e1844d941 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:48.449331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9b44-01d2-4af7-8945-b23e1844d941 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:48.451498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 602/1206] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:44:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:44:48.456988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:48.457498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:48.457651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:48.476247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:44:48.477532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:44:48.556083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 605/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:44:48.579747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9b44-01d2-4af7-8945-b23e1844d941 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:48.703521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9b44-01d2-4af7-8945-b23e1844d941 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:48.743743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:48.744462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:48.744773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:48.745096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:48.756201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:44:48.761365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:48.762618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:48.764407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 603/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:44:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:48.769984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:48.814397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:48.879666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5a96f-a5a8-453e-9191-c9ae79b4af60 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:48.922495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:48.923187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:48.923521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:48.923799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:48.924572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:48.924920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:48.926537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:48.935156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 606/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:44:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:48.941879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:49.016777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:49.073625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793552-00b8-4d7a-a711-95f081dff4ae None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:49.110739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:49.111395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:49.111679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:49.111950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:49.122746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:49.124317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:49.140640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:49.141877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:49.142955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:49.142955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:49.142955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:49.142955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:49.148293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 604/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:49.148537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:49.289013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:49.292417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0705cf48-0f73-4ad3-b66c-bd6cbb4c9182 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:49.336276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:49.337096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:49.337437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:49.337823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:49.338753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:49.339158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:49.340640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:49.361161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 607/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:44:49.362446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:49.472820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3670f-0cf6-4a32-9e6c-7a5f2bac888a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:49.475182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:49.515466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:49.516625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:49.517119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:49.517793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] PATH_INFO: `/v3/users/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:49.527856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:49.529537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:44:49.545159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:49.546190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:49.547645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 605/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:49.552268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:49.668797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bbd0d-2618-4e49-b00a-23c13e6bb380 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:49.709770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:49.709770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:49.710759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:49.711080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:49.712322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:49.712673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:49.714606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:49.734723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 608/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:44:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:49.740716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:49.852160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3c6a4-0c31-4d83-b1be-c8f36e46a87c None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:49.890436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:49.891126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:49.891472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:49.891747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:49.932601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:49.985794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:49.987447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:50.099513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 606/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:44:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:44:50.105360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:50.128873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:50.245240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1f28e-599f-4934-b07a-131411fa97c8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:50.291831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:50.292587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:50.292871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:50.293144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:50.304993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:50.306549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:50.326022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:50.328157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:50.329460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:50.329460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:50.329460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:50.329460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:50.330010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 609/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:44:50.336060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:50.415700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:50.462892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1673353-36a0-488a-adca-00fe97d4ec81 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:50.502219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:50.502916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:50.503115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:50.503452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:50.504168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:50.504533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:50.506984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:50.523944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 607/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:44:50.525422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:50.603724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:50.638804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75def793-af2c-4eef-ad0f-746a7059a9f5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:50.680664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:50.681831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:50.682138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:50.682629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:50.693680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:50.695491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:44:50.714155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:50.715180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:50.716191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 610/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:44:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:50.721831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:50.830969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc0551-3b15-4ae7-9480-b2a9e1d4c27f None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:50.855748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:50.869917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:50.870736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:50.871118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:50.871551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:50.872998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:50.873393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:50.875111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa464dca-04af-4427-99e7-af6f48a64235 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:50.893206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 608/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:44:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:50.894438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:51.011945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3075e19b-d19e-484b-a70c-a89bc196fa14 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:51.050237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:51.058418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:51.059536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:51.060761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:51.061157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:51.062083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=72012ef48a864233970d32552c403409) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:51.062507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:51.064042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281eba4-12f4-4c7b-9897-64d191525a21 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:51.070090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 611/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:44:50 2022] GET /identity/v3/role_assignments?user.id=72012ef48a864233970d32552c403409&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:53.257704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1db8-3f72-4f30-991f-d65954d17d5b None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:53.258440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1db8-3f72-4f30-991f-d65954d17d5b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:53.258440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f1db8-3f72-4f30-991f-d65954d17d5b None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:53.259565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 609/1220] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:44:53.266280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:53.266772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:53.266865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:53.285600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:44:53.286795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:44:53.358779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 612/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:44:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:44:53.381315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f1db8-3f72-4f30-991f-d65954d17d5b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:53.515585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f1db8-3f72-4f30-991f-d65954d17d5b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:53.559045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:53.561046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:53.561271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:53.561516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] PATH_INFO: `/v3/roles/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:53.573720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:44:53.579175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:53.580202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:53.581163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 610/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:44:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:44:53.587539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:53.707973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc9620-7a9a-455b-a85c-4ba0b23ffe2e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:53.739940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:53.753971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:53.754674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:53.754982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:53.755285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:53.756061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] RBAC: Adding query filter params (name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:53.756394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:53.757775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:53.767287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 613/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:44:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:53.773233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:53.822036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:53.884500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78014a0b-a922-40b3-9b86-75618203918d None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:53.921990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:53.922702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:53.922982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:53.923267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:53.935810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:53.937552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:53.954879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:53.956000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:53.957290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:53.957290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:53.957290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:53.957290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:53.957290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 611/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:44:53.962135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.067921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c11846-b454-4bba-9d0a-30034c7a9c4a None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:54.103142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:54.103769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:54.104042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:54.104330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:54.105024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:54.105326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:54.106801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:54.119073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 614/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:54.125104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.234383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:54.241524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7033c5-92af-452d-8e94-fb58046f4b31 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:54.289172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:54.290130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:54.290130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:54.290413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] PATH_INFO: `/v3/users/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:54.298998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:54.300565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:44:54.313689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:54.314860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:54.315795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 612/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:54.321420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.374141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:54.436798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1167e-78d3-434f-91b3-da9da31fb2db None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:54.476715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:54.477331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:54.477614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:54.477886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:54.478674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:54.478979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:54.480429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:54.498542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 615/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:44:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:54.504098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.610809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ffa88-f53c-4fad-8d8f-365610ff6cfc None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:54.649952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:54.650588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:54.650865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:54.651135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:54.661904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:54.663526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:54.678241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:54.679192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:54.680085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:54.680085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:54.680085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:54.680085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:54.680085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 613/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:54.686117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.829877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdbf6a-0010-4570-b8ee-5a7ae82bbd63 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:54.832289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:54.870709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:54.871700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:54.872901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:54.873354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:54.874644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:54.875220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:54.878016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:54.897602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 616/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:54.898830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:54.908613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:55.013869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0ac75-ca61-42a1-adbf-ada2d245e7ff None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:55.052670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:55.053460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:55.053767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:55.054041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:55.062542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:55.064141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:44:55.078216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:55.079401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:55.080884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:55.084398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 614/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:44:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 20:44:55.085981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:55.210464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdb323-6f41-47a1-ad20-48050d33151f None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:55.253597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:55.254280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:55.254568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:55.254837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:55.255617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:55.256111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:55.257631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42aa53d-cc6a-46af-a33f-6e5ac9cec342 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:55.271576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 617/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:55.278168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:55.354186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:55.392177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0873588e-c18d-4308-bb31-34d7f0f9fa65 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:55.429617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:55.430280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:55.430563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:55.430885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/72012ef48a864233970d32552c403409/roles/78496973911641848ac1ee3031c692e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:55.446888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=72012ef48a864233970d32552c403409, role_id=78496973911641848ac1ee3031c692e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:55.448504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b985b-d783-4d70-aa0a-214ef93c7529 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:55.464429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 615/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:44:55 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/72012ef48a864233970d32552c403409/roles/78496973911641848ac1ee3031c692e6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:44:57.539879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbffa8-462c-43f3-9479-48ce64f122ea None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:57.540420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbffa8-462c-43f3-9479-48ce64f122ea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:57.540420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbffa8-462c-43f3-9479-48ce64f122ea None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:57.541859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 618/1233] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:44:57.548219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:57.548710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:57.548799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:57.573817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:44:57.575747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:44:57.668372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 616/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:44:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 20:44:57.684735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbffa8-462c-43f3-9479-48ce64f122ea None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:57.711207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:57.793002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbffa8-462c-43f3-9479-48ce64f122ea None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:57.828679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:57.829469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:57.829748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:57.830019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:57.842920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:44:57.847808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:57.848938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:57.855001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 619/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Thu Aug 4 20:44:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:57.855851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:57.906614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:57.968359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb366b4f-f00d-43e7-90fa-08c04df20bc1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.003159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.003780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.004063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.004353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.005079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] RBAC: Adding query filter params (name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:58.005380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.006754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:58.016279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 617/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Thu Aug 4 20:44:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:58.022297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:58.114366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:58.133271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1d78c-5ad6-4cfa-84ad-ab838dfe6bf4 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.168534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.169801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.170179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.170457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.181741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.183221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:58.200440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:58.207116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:58.208012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:58.208012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:58.208012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:58.208012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:58.208012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 620/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:58.208308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:58.323471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90743f93-5d2d-49fe-9b05-51b70e667453 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.333773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:58.363241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.363898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.364237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.364582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.365296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:58.365606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.367069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:58.379963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 618/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:58.385691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:58.498933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe05ea-a870-4ca5-90ad-a0c8de4394b6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.540562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.541221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.541617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.542040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.546301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:58.552549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.554322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:44:58.566499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:58.567920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:44:58.568936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 621/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:58.573756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:58.688163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9dd68b-527b-4381-9db0-78016730910a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.728067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.728727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.729033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.729305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.730092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:58.730393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.731928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:58.752122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 619/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:44:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:44:58.759201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:58.867808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574cc59-fe62-4f34-8c2c-6e89a716c3c0 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:58.904513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:58.905223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:58.905547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:58.905865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:58.974431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:58.983851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:58.985344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:59.095785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 622/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:44:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:44:59.101697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:59.131967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:59.217889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd3672-e354-4ade-9258-fbce270042db None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:59.256593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:59.257223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:59.257533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:59.257842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:59.269374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:59.270870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:44:59.287059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:59.288312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:59.289452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:44:59.289452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:44:59.289452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:44:59.289452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:59.289452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 620/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:59.295488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:59.422759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:59.433136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5af36d-107c-45c6-9961-456766225270 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:59.473848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:59.474462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:59.474739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:59.475006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:59.475682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:59.475978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:59.477367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:59.490923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 623/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:59.496490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:59.601068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:44:59.628046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0348db-8f47-44e1-8c09-f05f3d050cf2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:59.670013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:59.670660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:59.670946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:59.671225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:59.680199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:59.681928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:44:59.695345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:44:59.696542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:44:59.697470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 621/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:44:59.703503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:59.808743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356605f-7de9-410d-b779-a6c97ddf46a8 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:44:59.844379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:44:59.844994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:44:59.845271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:44:59.845537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:44:59.846271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:44:59.846572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:44:59.847988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d94a96-57b5-4109-82e9-603684263f85 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:44:59.863232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 624/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:44:59.869434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:44:59.986265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a7d1c-0439-40df-8676-3d900661cb87 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:00.030986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:00.031663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:00.031937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:00.032258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:00.033131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] RBAC: Adding query filter params (role.id=78496973911641848ac1ee3031c692e6, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=72012ef48a864233970d32552c403409) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:00.033448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:00.035126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a2a88-9daa-4d9b-9b96-769363cc31e4 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:00.042107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 622/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:44:59 2022] GET /identity/v3/role_assignments?user.id=72012ef48a864233970d32552c403409&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=78496973911641848ac1ee3031c692e6 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:02.343852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a915cf-7841-40a3-8ad5-21f67c4415f4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:02.344491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a915cf-7841-40a3-8ad5-21f67c4415f4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:02.344491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a915cf-7841-40a3-8ad5-21f67c4415f4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:02.345790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 625/1247] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:45:02.352392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:02.352856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:02.352952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:02.361062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:02.374422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:02.375741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:02.451427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 623/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:02.476273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a915cf-7841-40a3-8ad5-21f67c4415f4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:02.496324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:02.597407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a915cf-7841-40a3-8ad5-21f67c4415f4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:02.639096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:02.639831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:02.640162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:02.640554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:02.651969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:45:02.658611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:02.659823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:02.660996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 626/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:02.666354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:02.781994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac723-e4a6-4771-a29d-c23ce540d503 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:02.813561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:02.819075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:02.819775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:02.820133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:02.820512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:02.821297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:02.821674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:02.823171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:02.833012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 624/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:45:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:02.838887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:02.925174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:02.953321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75eb73b-6b9f-4a50-9e24-7867326fe830 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:02.999828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:03.000726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:03.001122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:03.001505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:03.015394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:03.017027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:03.040124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:03.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:03.042290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:03.042290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:03.042290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:03.042290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:03.042290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 627/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:03.048649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:03.173144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce11fb-7db4-4018-9e8e-85a8cfdeab7e None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:03.214089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:03.214860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:03.215157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:03.215515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:03.216620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:03.217039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:03.218867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:03.234682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 625/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:03.240822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:03.359340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec13dd-ffad-4fee-ac0f-83762d59e4d4 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:03.389609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:03.399017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:03.399665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:03.399948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:03.400227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:03.409627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:03.411088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:45:03.423918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:03.425251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:03.426984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 628/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:03.431437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:03.494401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:03.548910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab434010-c667-43a0-8efb-22a3d6832c32 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:03.585930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:03.586559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:03.586874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:03.587141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:03.587908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:03.588218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:03.589675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:03.608571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 626/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:45:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:03.614857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:03.735257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce19697b-0f5f-4ca2-ab13-7184438d35b6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:03.773015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:03.773660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:03.773940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:03.774231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:03.847303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:03.862750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:03.864267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:03.978240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 629/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:45:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:45:03.984648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:04.105780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37538b-0831-481b-a8dc-b794cc298151 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:04.123292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:04.147960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:04.148726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:04.149009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:04.149279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:04.161103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:04.162716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:04.181964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:04.183383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:04.184314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:04.184314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:04.184314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:04.184314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:04.184314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 627/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:04.190293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:04.307642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:04.323973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70d2d4-6700-4734-9778-c31051066708 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:04.368059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:04.368871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:04.368963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:04.369224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:04.370046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:04.370350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:04.372412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:04.384682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 630/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:04.391211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:04.511402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abc2c94-242f-4ec9-b730-6ef8a9414a72 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:04.549492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:04.550146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:04.550453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:04.550714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:04.560553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:04.562064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:45:04.578176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:04.579291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:04.580459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 628/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:04.585561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:04.694002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635884ee-02e8-4ce4-87b4-7d1f053fce93 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:04.730491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:04.731599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:04.731938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:04.732212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:04.732983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:04.733281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:04.734710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151e655-8764-4c4a-a347-8e1e4f3dda96 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:04.749827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 631/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:04.755993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:04.786389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:04.874363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c74cee-9fb2-4d8b-9d83-d746a43da2f5 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:04.912932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:04.913650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:04.914004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:04.914352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:04.915246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=72012ef48a864233970d32552c403409) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:04.915555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:04.917026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e92fa-920a-492a-a24a-51820e0e6f6d None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:04.932379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 629/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:45:04 2022] GET /identity/v3/role_assignments?user.id=72012ef48a864233970d32552c403409&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:07.010739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd515f-305a-43fd-a662-edba8f1dc978 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.011251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd515f-305a-43fd-a662-edba8f1dc978 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.011251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd515f-305a-43fd-a662-edba8f1dc978 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.012600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 632/1261] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:07.017864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.018242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.018429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.036402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:07.037605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:07.102971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 630/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:07.126179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd515f-305a-43fd-a662-edba8f1dc978 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:07.136841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:07.232961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd515f-305a-43fd-a662-edba8f1dc978 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:07.265952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:07.266594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.266877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.267154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.277077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:45:07.282833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:07.283897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:07.285164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 633/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:07.290059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:07.308683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:07.386490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fab7cc-f152-421c-9953-c13c7de22c2e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:07.421257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:07.422315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.422818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.423117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.423825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:07.424160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:07.425586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:07.433754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 631/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:07.439450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:07.504780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:07.539387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b265df0a-f884-41a7-98f9-ce8720e43462 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:07.572560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:07.573190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.573470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.573740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.584140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:07.585610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:07.600920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:07.601952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:07.603131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:07.603131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:07.603131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:07.603131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:07.603131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 634/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:07.607851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:07.686575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:07.706341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082d874-37c0-46fb-9c87-3cc3bfe5e8ea None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:07.741724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:07.742335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.742618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.742890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.743566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:07.743868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:07.745250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:07.757094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 632/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:07.763246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:07.859655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ac668-9111-4abb-817a-bc52ebd30047 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:07.878141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:07.892225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:07.892882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:07.893158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:07.893425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:07.901374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:07.902822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:45:07.940092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:07.941310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:07.942458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 635/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:07.948998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.058062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b437b1-f98b-4f43-b993-cd38ad19eb36 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.093717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.094328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.094609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.094886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.095665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:08.095971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.097426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.105709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:08.114954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 633/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:45:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:08.120584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.230939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41db6103-09c1-4641-a927-f43181e847d4 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.264641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.265237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.265516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.265790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.276543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.278602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:08.295575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:08.296722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:08.297777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:08.297777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:08.297777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:08.297777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:08.297777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 636/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:08.303831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.411448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8479b5-df86-4969-9d5f-1e0534f9e6ad None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.447232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.447847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.448129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.448428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.449113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:08.449412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.451816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.462839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 634/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:08.469155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.500517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:08.568188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbf566-9eb8-497a-a7c3-c5cae38ce4a6 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.600740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.601359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.601635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.601903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.608228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.609686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:45:08.620579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:08.621693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:08.622839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 637/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:45:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:08.627706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.692131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:08.734994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b3680-0801-4d51-bb87-ebe55405a734 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.768442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.769052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.769337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.769612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.770354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:08.770667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.772093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b4c74-5776-4326-bdda-913b2528bbfa None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.783913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 635/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:45:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:08.789899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:08.892962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:08.902250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b211-d736-4b55-8c3c-0bd5fc008fb1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:08.936460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:08.937119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:08.937402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:08.937674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] PATH_INFO: `/v3/projects/07b38a76369e4d47bac7906590383b26/users/72012ef48a864233970d32552c403409/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:08.939548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] RBAC: Authorizing `identity:create_grant(project_id=07b38a76369e4d47bac7906590383b26, user_id=72012ef48a864233970d32552c403409, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:08.941077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ed40a-9079-4dff-92fc-7568b557cf1e None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:08.954985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 638/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Thu Aug 4 20:45:08 2022] PUT /identity/v3/projects/07b38a76369e4d47bac7906590383b26/users/72012ef48a864233970d32552c403409/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:45:10.976631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dedfa1-914f-4fc9-8d8c-58a062a65bfb None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:10.977229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dedfa1-914f-4fc9-8d8c-58a062a65bfb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:10.977229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dedfa1-914f-4fc9-8d8c-58a062a65bfb None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:10.978440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 636/1274] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:10.983732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:10.984171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:10.984271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.002750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:11.003930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:11.068886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 639/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:11.092235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dedfa1-914f-4fc9-8d8c-58a062a65bfb None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.195054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dedfa1-914f-4fc9-8d8c-58a062a65bfb None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:11.215441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:11.232699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:11.233279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:11.233564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:11.233841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.243650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 20:45:11.249627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:11.250813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:11.251724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 637/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:11.256906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.336324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:11.354598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e5640-becf-4a48-8121-474ab3cc084f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:11.389067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:11.389677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:11.389959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:11.390232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.390933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:11.391235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:11.392625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:11.400419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 640/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:11.406765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.507846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e623661-614b-43bf-8122-ee14cf9f1253 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:11.540935 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:11.541573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:11.541857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:11.542134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.553093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:11.554833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:11.570684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:11.571763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:11.572954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:11.572954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:11.572954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:11.572954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:11.572954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 638/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:11.577821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.675163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42e7e0-feee-4654-b5e8-10893ceac9f1 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:11.707255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:11.707864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:11.708150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:11.708462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.709144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:11.709452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:11.710812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:11.722811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 641/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:11.728465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.759571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:11.829026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24201ef1-80ab-4bdb-852a-2c0d5359f4e2 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:11.862778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:11.863400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:11.863679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:11.863950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:11.871137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:11.872743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 20:45:11.883546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:11.884521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:11.885557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 639/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:11.890808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:11.894500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:11.988704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9222f54-137a-4220-b484-62c0a7b01f5b None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.021215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.021816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.022092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.022359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.023128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:12.023428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.024859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.041782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 642/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Thu Aug 4 20:45:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:12.048071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:12.135384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:12.145723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ebb93-3f09-4e10-91de-fa6d5f23bd36 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.181755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.182826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.183111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.183458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.255027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.256502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.343585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:12.353966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 640/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 20:45:12.359787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:12.419269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:12.458318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8416a26-9d88-4084-b5ac-06227179db7d None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.491727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.492368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.492658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.492926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.503172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.504635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 20:45:12.519776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:12.520967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:12.521881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 20:45:12.521881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 20:45:12.521881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 20:45:12.521881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:12.521881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 643/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:12.526763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:12.623392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ba0d1-b076-4214-a9d3-2ccc3da4a5a6 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.656365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.656984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.657261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.657530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.658205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:12.658501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.659845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.670762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 641/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:12.676704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:12.775062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c824515e-3dfb-4fcd-84e9-7d3926906370 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.788511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:12.810134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.810766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.811050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.811327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] PATH_INFO: `/v3/projects/service` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.817863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.819357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 20:45:12.831428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:12.832512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: ERROR keystone.server.flask.application Aug 04 20:45:12.833607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 644/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:12.838405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:12.888571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:12.938839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb90ca-4eca-4340-a3e3-3c8314c6bcbb None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:12.971489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:12.972095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:12.972405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:12.972680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:12.973417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:12.973722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:12.975152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3bf50-96a1-4a75-b755-63212cc54768 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:12.987337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 642/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:12.993394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:13.091659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf946a-c852-4a12-aec0-83c4d0499110 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:13.125558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:13.126163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:13.126439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:13.126708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:13.127477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] RBAC: Adding query filter params (role.id=8b2191da1ad1452b8f5f5353376e06bb, scope.project.id=07b38a76369e4d47bac7906590383b26, user.id=72012ef48a864233970d32552c403409) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:13.127777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:13.129216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3cd4c-2039-46b6-b816-6fa4db632326 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:13.134815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 645/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Thu Aug 4 20:45:12 2022] GET /identity/v3/role_assignments?user.id=72012ef48a864233970d32552c403409&scope.project.id=07b38a76369e4d47bac7906590383b26&role.id=8b2191da1ad1452b8f5f5353376e06bb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:45:15.142640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad7f2-b669-4227-a793-7fbc3a59728b None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:15.143267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad7f2-b669-4227-a793-7fbc3a59728b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:15.143267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdad7f2-b669-4227-a793-7fbc3a59728b None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:15.144612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 643/1288] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:15.149871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:15.150235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:15.150427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:15.169403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:15.170626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:15.203511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:15.250164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 646/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:15.272152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdad7f2-b669-4227-a793-7fbc3a59728b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:15.370299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdad7f2-b669-4227-a793-7fbc3a59728b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:15.380506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:15.404813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:15.405368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:15.405646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:15.405917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:15.406528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:15.407880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:15.416415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:15.417635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:15.418726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 644/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:45:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:15.424013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:15.521154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8209da0-86d7-4c0e-909f-3e7a6e5a702b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:15.555588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:15.556200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:15.556513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:15.556792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:15.557469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] RBAC: Adding query filter params (name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:15.557771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:15.559670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfbc523-b1f6-443c-aa75-eddf324e4f6a None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:15.564428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 647/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Aug 4 20:45:15 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:15.570132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:15.666798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97d52-256e-49af-8d1f-43f1fb3ba3c8 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:15.699313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:15.700174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:15.700489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:15.700762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] PATH_INFO: `/v3/services` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:15.701426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] RBAC: Adding query filter params (type=placement) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:15.701726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:15.703075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec75e1a-c98c-4c80-a811-d14666e09195 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:15.707475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 645/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Aug 4 20:45:15 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:17.728907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494773c-f460-463e-acaf-09e6358f1a43 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:17.729424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494773c-f460-463e-acaf-09e6358f1a43 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:17.729547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494773c-f460-463e-acaf-09e6358f1a43 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:17.730836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 648/1293] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:17.736101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89b1c-2151-444c-8ad4-b9f6bd97a80c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:17.736541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89b1c-2151-444c-8ad4-b9f6bd97a80c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:17.736697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f89b1c-2151-444c-8ad4-b9f6bd97a80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:17.754948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-25f89b1c-2151-444c-8ad4-b9f6bd97a80c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:17.756139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-25f89b1c-2151-444c-8ad4-b9f6bd97a80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:17.785711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:17.822890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 646/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:17.845626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494773c-f460-463e-acaf-09e6358f1a43 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:17.875978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:17.948818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494773c-f460-463e-acaf-09e6358f1a43 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:17.981938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:17.982538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:17.982815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:17.983083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:17.983761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:17.985221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12dd7b9-70af-42d9-8be1-e02c542366b0 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:17.999101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 649/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Thu Aug 4 20:45:17 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:45:20.010522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77826-6f21-4058-983b-f6e85effaff2 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.011034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77826-6f21-4058-983b-f6e85effaff2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.011034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd77826-6f21-4058-983b-f6e85effaff2 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.012382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 647/1296] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:20.018215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.018677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.018770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.036281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:20.037668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:20.109148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 650/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:20.132662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd77826-6f21-4058-983b-f6e85effaff2 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:20.247454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd77826-6f21-4058-983b-f6e85effaff2 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:20.281071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:20.281715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.282001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.282280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] PATH_INFO: `/v3/services/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.282901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:20.284276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:20.293494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:20.295044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:20.295905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 648/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:45:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:20.301008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:20.319177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:20.398708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508129d6-df8a-4f82-aac3-ea8b452615fd None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:20.432319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:20.432955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.433235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.433506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.434751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:20.435080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:20.436463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:20.441452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 651/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Aug 4 20:45:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:20.447511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:20.478287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:20.549228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4caee2-0cad-4d98-a0b5-091472d2a98a None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:20.583429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:20.584045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.584328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.584637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.585387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] RBAC: Adding query filter params (interface=public, service_id=47c628f1c62745c988bcad9ff4f03108, region_id=RegionOne) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:20.585697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:20.587068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99758d-09dc-4954-b747-70e4c72e9b41 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:20.592250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 649/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Thu Aug 4 20:45:20 2022] GET /identity/v3/endpoints?service_id=47c628f1c62745c988bcad9ff4f03108&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:20.597858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:20.678306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:20.696080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a217763-3380-455c-8fd9-650b7ae171b5 None admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:20.729295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:20.729905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:20.730188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:20.730487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:20.731732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:20.733113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e58024-0cd8-46a2-911e-b4c6ecdbec43 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:20.739753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 652/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Thu Aug 4 20:45:20 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:45:22.713386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4c78-920e-4b9c-8de4-58275f032127 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:22.713936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4c78-920e-4b9c-8de4-58275f032127 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:22.713936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6a4c78-920e-4b9c-8de4-58275f032127 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:22.715442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 650/1302] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:45:22.720644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:22.721044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:22.721191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:22.739797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:45:22.741012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:45:22.807740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 653/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:45:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 20:45:22.831070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4c78-920e-4b9c-8de4-58275f032127 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:22.925242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:22.932165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6a4c78-920e-4b9c-8de4-58275f032127 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:22.964557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:22.965748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:22.966000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:22.966272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] PATH_INFO: `/v3/services/placement` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:22.966889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:22.968262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 20:45:22.976977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 20:45:22.978126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application with Lock( Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self._enter() Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application return self.creator() Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application created_value = creator( Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: ERROR keystone.server.flask.application Aug 04 20:45:22.978987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 651/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Thu Aug 4 20:45:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 20:45:22.984761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:23.058762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:23.081107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa3f08-5e16-4376-9e59-7bc77980bd0d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:23.114837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:23.115447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:23.115731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:23.116009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] PATH_INFO: `/v3/services` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:23.116711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] RBAC: Adding query filter params (name=placement) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:45:23.117017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:23.118382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e77fe8-76cb-491b-b4ec-6452639ebd04 None admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:23.123476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 654/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Thu Aug 4 20:45:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:45:23.129264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:45:23.236293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12edc7b-0e99-4eef-9b0a-e34403eeda90 None admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:45:23.268868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:45:23.269477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:45:23.269759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:45:23.270032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] PATH_INFO: `/v3/endpoints` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:45:23.270737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:45:23.272097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c6fba-fff7-4bc7-97e8-53c8d5c75226 None admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:45:23.291205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 652/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:45:23 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:45:33.120538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=30 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:45:33.287024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:27.349600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5dfb-942e-4c2b-8044-f10a1084f914 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:27.350093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5dfb-942e-4c2b-8044-f10a1084f914 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:27.350560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa5dfb-942e-4c2b-8044-f10a1084f914 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:27.352801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 655/1307] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:27.358807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8be539-d382-47c5-872e-6f0ffea9d16e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:27.359256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8be539-d382-47c5-872e-6f0ffea9d16e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:27.359348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8be539-d382-47c5-872e-6f0ffea9d16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:27.509100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1e8be539-d382-47c5-872e-6f0ffea9d16e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:27.510732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1e8be539-d382-47c5-872e-6f0ffea9d16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:27.703300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 653/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:27.763435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:27.764100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:27.764629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:27.766581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 656/1309] 10.222.0.52 () {54 vars in 965 bytes} [Thu Aug 4 20:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:27.773734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d6bc-57b0-41a7-8357-336c9f288d30 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:27.774097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d6bc-57b0-41a7-8357-336c9f288d30 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:27.774327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32d6bc-57b0-41a7-8357-336c9f288d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:27.878784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5d32d6bc-57b0-41a7-8357-336c9f288d30 None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:27.880056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5d32d6bc-57b0-41a7-8357-336c9f288d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:27.912481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:27.981207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 654/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Aug 4 20:46:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:27.990280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:27.992088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:28.117289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e410a0-e09c-40d8-9241-be31044b46ce None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:28.197252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:28.197782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:28.198095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:28.198380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:28.200020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:28.259913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:28.320890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:28.322738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e5bbe-856f-4b54-bb76-d715116282de service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:28.363768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 657/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:30.727110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13ca2-a528-4b81-9995-ba873b6b07fb None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:30.727641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13ca2-a528-4b81-9995-ba873b6b07fb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:30.727641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b13ca2-a528-4b81-9995-ba873b6b07fb None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:30.729044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 655/1312] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:30.734387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2420-7841-4f07-9571-8e3a5f913102 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:30.734824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2420-7841-4f07-9571-8e3a5f913102 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:30.734938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a2420-7841-4f07-9571-8e3a5f913102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:30.818507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d12a2420-7841-4f07-9571-8e3a5f913102 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:30.819640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d12a2420-7841-4f07-9571-8e3a5f913102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:30.922099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 658/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:30.972655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:30.972938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:30.973212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:30.974561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 656/1314] 10.222.0.52 () {54 vars in 965 bytes} [Thu Aug 4 20:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:30.980895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d5d-00ab-4adb-8bd2-c86a34f9ea46 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:30.981190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d5d-00ab-4adb-8bd2-c86a34f9ea46 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:30.981473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d5d-00ab-4adb-8bd2-c86a34f9ea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:30.993217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:31.057155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-72552d5d-00ab-4adb-8bd2-c86a34f9ea46 None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:31.058411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-72552d5d-00ab-4adb-8bd2-c86a34f9ea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:31.135020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 659/1315] 10.222.0.52 () {58 vars in 1062 bytes} [Thu Aug 4 20:46:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:31.143900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:31.145468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:31.272609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a73c7-022e-4a09-9ccd-94379bf0fdff None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:31.344284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:31.346908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:31.347756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:31.348225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:31.348721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:31.350352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:31.459805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:31.461189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0067e8-c26a-4054-bdfb-11d6d1d86339 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:31.480992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 657/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:34.005660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:34.006191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:34.006191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:34.007465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 660/1317] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:34.013351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48453e-1ebd-4594-9b6f-32ab71054c2d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:34.013749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48453e-1ebd-4594-9b6f-32ab71054c2d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:34.013895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48453e-1ebd-4594-9b6f-32ab71054c2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:34.078721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-aa48453e-1ebd-4594-9b6f-32ab71054c2d None None] MFA Rules not processed for user `5890e1b139df49a09f8663306e5ab7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:34.079924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-aa48453e-1ebd-4594-9b6f-32ab71054c2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:34.130510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:34.176878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 658/1318] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:34.218663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:34.220219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:34.324161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f83d96-0dda-4d09-b5c6-951508303d02 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:34.326962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:34.396360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:34.396981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:34.397260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:34.397532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:34.399124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:34.504095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:34.505590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87670381-bb62-444b-b7b9-198a98751d6b service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:34.524264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 661/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:37.110835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:37.111345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:37.111345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:37.112685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 659/1320] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:37.117905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d90-5a51-4e8b-855e-66da89dbaa23 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:37.118354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d90-5a51-4e8b-855e-66da89dbaa23 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:37.118451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62908d90-5a51-4e8b-855e-66da89dbaa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:37.159939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:37.186771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-62908d90-5a51-4e8b-855e-66da89dbaa23 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:37.188099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-62908d90-5a51-4e8b-855e-66da89dbaa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:37.283131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 662/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:37.323182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:37.330215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:37.428436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae66548-b5ef-4809-97dc-9efba6495338 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:37.500739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:37.501356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:37.501640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:37.501917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:37.503617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:37.511358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:37.615201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:37.616693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a7369-719a-48ec-a633-d25b236ccf00 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:37.635408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 660/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:37 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:40.320510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:40.321081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:40.321081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:40.322694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 663/1323] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:46:40.327849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375c25-1ff2-4001-a8fe-8c4627bcf0b8 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:40.328297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375c25-1ff2-4001-a8fe-8c4627bcf0b8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:40.328529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e375c25-1ff2-4001-a8fe-8c4627bcf0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:40.397268 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7e375c25-1ff2-4001-a8fe-8c4627bcf0b8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:40.398474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7e375c25-1ff2-4001-a8fe-8c4627bcf0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:40.460133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:40.508237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 661/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:40.546483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:40.548089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:40.664048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33fca5-95d1-45e4-8051-46e0c40ef9ae None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:40.677236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:40.742629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:40.743786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:40.744081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:40.744506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:40.746020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:40.857446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:40.858917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b884ec-62fe-4794-b23b-027805f3557f service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:40.878682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 664/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:43.001465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:43.002026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:43.002026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:43.003310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 662/1326] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:43.008597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c02d6-042a-488a-be65-b7b3e6a16700 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:43.009565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c02d6-042a-488a-be65-b7b3e6a16700 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:43.009709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c02d6-042a-488a-be65-b7b3e6a16700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:43.059570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:43.078543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-426c02d6-042a-488a-be65-b7b3e6a16700 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:43.079738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-426c02d6-042a-488a-be65-b7b3e6a16700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:43.194627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 665/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:43.232271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:43.233954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:43.358721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509b330-5cb1-4435-afb1-c83a8a73f384 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:43.437985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:43.438616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:43.438901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:43.439177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:43.441025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:43.461462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:43.581413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:43.582867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fac3e4-5d4a-48e9-8281-08fb04359f6e service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:43.602747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 663/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:45.685443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:45.686026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:45.686026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:45.687344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 666/1329] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:45.692492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f3fd6-2872-41cd-8e88-afe047bb4eed None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:45.692934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f3fd6-2872-41cd-8e88-afe047bb4eed None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:45.693038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f3fd6-2872-41cd-8e88-afe047bb4eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:45.760362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-217f3fd6-2872-41cd-8e88-afe047bb4eed None None] MFA Rules not processed for user `5890e1b139df49a09f8663306e5ab7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:45.761585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-217f3fd6-2872-41cd-8e88-afe047bb4eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:45.830036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:45.863925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 664/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:45 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:45.904981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:45.906641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:46.011132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc00c91-9445-407e-b85b-861a5a63b6f2 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:46.030622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:46.084841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:46.085368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:46.085648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:46.085924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:46.087615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:46.201605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:46.203033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d44c2-40db-43ee-a91b-aae7ed24fe9e service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:46.221450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 667/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:45 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:48.568325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:48.568856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:48.568950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:48.570300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 665/1332] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:48.575629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388eff-9b25-4871-bbbf-423f9295ad91 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:48.576069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388eff-9b25-4871-bbbf-423f9295ad91 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:48.576212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f388eff-9b25-4871-bbbf-423f9295ad91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:48.639099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:48.641978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6f388eff-9b25-4871-bbbf-423f9295ad91 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:48.643719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6f388eff-9b25-4871-bbbf-423f9295ad91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:48.741920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 668/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:48.779817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:48.781347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:48.885171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb13bf9-d66a-432c-ba54-49a99e576cdb None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:48.962628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:48.963244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:48.963533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:48.963808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:48.965474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:48.994335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:49.081802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:49.083247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fbac-0fed-46ec-9648-f34c777e1c05 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:49.101889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 666/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:51.155724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:51.156248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:51.156248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:51.157633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 669/1335] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:51.162867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891670c1-32b1-4a33-95a5-8f060333c25d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:51.163271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891670c1-32b1-4a33-95a5-8f060333c25d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:51.163433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891670c1-32b1-4a33-95a5-8f060333c25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:51.230760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-891670c1-32b1-4a33-95a5-8f060333c25d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:51.231973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-891670c1-32b1-4a33-95a5-8f060333c25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:51.306579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:51.329663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 667/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:51.371124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:51.372670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:51.475562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a3ea5-ae2a-4932-9f4a-5627cf0b9e0c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:51.504271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:51.550836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:51.551452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:51.551732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:51.552005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:51.553648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:51.666226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:51.667662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7581b0d3-7c83-4ba2-b276-6266efc48e94 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:51.686661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 670/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:46:54.345479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:54.346028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:54.346191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:54.347550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 668/1338] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:54.353430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5690-8aba-498b-ad32-ec74ce9d612f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:54.353834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5690-8aba-498b-ad32-ec74ce9d612f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:54.353983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd5690-8aba-498b-ad32-ec74ce9d612f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:54.418961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5ebd5690-8aba-498b-ad32-ec74ce9d612f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:54.420155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5ebd5690-8aba-498b-ad32-ec74ce9d612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:54.428437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:54.516754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 671/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:54.554339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:54.556636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:54.661033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56423d-96a5-4253-b5ae-d38dd7ad782d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:54.734260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:54.734870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:54.735150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:54.735425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:54.737065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:54.772254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:54.847240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:54.848705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26535f3a-e2d9-4a3e-b18f-aada586048a8 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:54.867455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 669/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:46:58.959930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:58.960542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:58.960542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:58.962693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 672/1341] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:46:58.967940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05343d-3adc-4968-b452-2d8c48c3366a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:58.968382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05343d-3adc-4968-b452-2d8c48c3366a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:58.968525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05343d-3adc-4968-b452-2d8c48c3366a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:59.034889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0d05343d-3adc-4968-b452-2d8c48c3366a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:46:59.036091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0d05343d-3adc-4968-b452-2d8c48c3366a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:59.122245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:59.143766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 670/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:46:59.181416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:46:59.183329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:59.291746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aace1fb-90fd-4978-886f-597da396fc94 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:46:59.330218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:46:59.367443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:46:59.368091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:46:59.368403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:46:59.368681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:46:59.370292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:46:59.481218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:46:59.482566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d936c3d1-9e35-4e51-8d27-5baa428c5eee service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:46:59.500741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 673/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:02.152261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:02.152863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:02.152863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:02.154189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 671/1344] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:02.159327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae2cde-87b4-42a0-8ccd-2aa26f4412a7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:02.159764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae2cde-87b4-42a0-8ccd-2aa26f4412a7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:02.159877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae2cde-87b4-42a0-8ccd-2aa26f4412a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:02.239170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-87ae2cde-87b4-42a0-8ccd-2aa26f4412a7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:02.240476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-87ae2cde-87b4-42a0-8ccd-2aa26f4412a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:02.260266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:02.347644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 674/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:02.386346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:02.387916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:02.495604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a3cbb-0d6d-49cf-8b22-6522408de6d1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:02.573831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:02.574454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:02.574737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:02.575012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:02.576660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:02.622186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:02.693065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:02.694510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420104e-2495-4c3f-8d1b-3806a31fdd4e service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:02.716559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 672/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:06.741588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:06.742235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:06.742235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:06.744855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 675/1347] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:06.750464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ed0c-3415-4eb2-81b8-501dbf388250 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:06.750984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ed0c-3415-4eb2-81b8-501dbf388250 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:06.751079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541ed0c-3415-4eb2-81b8-501dbf388250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:06.825014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d541ed0c-3415-4eb2-81b8-501dbf388250 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:06.826265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d541ed0c-3415-4eb2-81b8-501dbf388250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:06.937151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:06.944380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 673/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:06.978861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:06.980513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:07.092209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b083b8c-1ccd-4931-9039-0b55fab54220 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:07.151760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:07.200528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:07.201143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:07.201418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:07.201686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:07.203277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:07.328076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:07.329562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b32008d-fc95-426b-973d-befbbbb361bb service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:07.349120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 676/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:12.519606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c7c2-884d-45ed-bc38-19efe6daec8c None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:12.520124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c7c2-884d-45ed-bc38-19efe6daec8c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:12.520207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052c7c2-884d-45ed-bc38-19efe6daec8c None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:12.522055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 674/1350] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:12.527849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3024-051c-4731-8513-3ac36162917e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:12.528308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3024-051c-4731-8513-3ac36162917e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:12.528453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace3024-051c-4731-8513-3ac36162917e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:12.636552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9ace3024-051c-4731-8513-3ac36162917e None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:12.638127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9ace3024-051c-4731-8513-3ac36162917e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:12.682557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:12.806016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 677/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:47:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:17.110333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691865e-c951-4dd5-b7f2-8a5341fe6efb None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:17.110908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691865e-c951-4dd5-b7f2-8a5341fe6efb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:17.110908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e691865e-c951-4dd5-b7f2-8a5341fe6efb None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:17.112219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 675/1352] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:17.118730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb3f5-d49c-41ac-8b29-4eabc5e44d63 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:17.119273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb3f5-d49c-41ac-8b29-4eabc5e44d63 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:17.119273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1fb3f5-d49c-41ac-8b29-4eabc5e44d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:17.215309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-bc1fb3f5-d49c-41ac-8b29-4eabc5e44d63 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:17.216505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bc1fb3f5-d49c-41ac-8b29-4eabc5e44d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:17.310744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:17.372207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 678/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:47:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:18.629013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511822d-cb09-41ee-9d0a-a800cdb4d3af None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:18.629548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511822d-cb09-41ee-9d0a-a800cdb4d3af None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:18.629644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511822d-cb09-41ee-9d0a-a800cdb4d3af None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:18.631032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 676/1354] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:18.636731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a275b3-f318-41c9-b832-73c5d235fb88 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:18.637310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a275b3-f318-41c9-b832-73c5d235fb88 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:18.637310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a275b3-f318-41c9-b832-73c5d235fb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:18.721741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-92a275b3-f318-41c9-b832-73c5d235fb88 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:18.722991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-92a275b3-f318-41c9-b832-73c5d235fb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:18.838956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 679/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:47:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:22.353894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841aacb-d1ce-43a7-b407-8963dd90f561 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:22.354782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841aacb-d1ce-43a7-b407-8963dd90f561 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:22.355066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841aacb-d1ce-43a7-b407-8963dd90f561 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:22.358135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 677/1356] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:22.362447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4651e-941b-44d5-92ee-74873141f98f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:22.363244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4651e-941b-44d5-92ee-74873141f98f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:22.363676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4651e-941b-44d5-92ee-74873141f98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:22.425642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:22.469849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0db4651e-941b-44d5-92ee-74873141f98f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:22.471261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0db4651e-941b-44d5-92ee-74873141f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:22.582382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 680/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:22.639211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:22.639920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:22.640416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:22.642480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 678/1358] 10.222.0.52 () {54 vars in 962 bytes} [Thu Aug 4 20:47:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:22.653762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ac8b1-146b-41fe-994b-919dd8f51683 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:22.654439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ac8b1-146b-41fe-994b-919dd8f51683 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:22.655070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ac8b1-146b-41fe-994b-919dd8f51683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:22.743962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1e1ac8b1-146b-41fe-994b-919dd8f51683 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:22.745203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1e1ac8b1-146b-41fe-994b-919dd8f51683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:22.799723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:22.876477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 681/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Aug 4 20:47:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:22.884982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:22.886563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:23.048111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8e220-4947-4116-820a-96692fb4e7bc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:23.125796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:23.171835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:23.172583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:23.172977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:23.173362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:23.175502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:23.279256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:23.280902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248f3c6-07ee-48c5-8007-6bccc6c9b819 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:23.301505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 679/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:47:25.582415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:25.582859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:25.582859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:25.584620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 682/1361] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:47:25.590940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37099d-b2b3-4631-92b0-a1f02eb53144 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:25.591482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37099d-b2b3-4631-92b0-a1f02eb53144 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:25.591482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37099d-b2b3-4631-92b0-a1f02eb53144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:25.666700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7a37099d-b2b3-4631-92b0-a1f02eb53144 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:25.667822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7a37099d-b2b3-4631-92b0-a1f02eb53144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:25.709051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:25.776581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 680/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:25.823490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:25.825903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:26.020142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99caa11-58e4-4234-b57e-96142869abc4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:26.031246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:26.190146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:26.191224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:26.191667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:26.192103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:26.195496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:26.322315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:26.323945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959f27c-2c06-4b56-bed0-a07b9e315b40 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:26.343808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 683/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:28.725464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2114c-119f-454a-88ea-83fde52e6b76 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:28.725979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2114c-119f-454a-88ea-83fde52e6b76 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:28.726425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2114c-119f-454a-88ea-83fde52e6b76 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:28.728685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 681/1364] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:28.734362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de819-c4fe-477e-9b1c-e9b6ab7465b6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:28.734864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de819-c4fe-477e-9b1c-e9b6ab7465b6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:28.735157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152de819-c4fe-477e-9b1c-e9b6ab7465b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:28.752583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:28.816330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-152de819-c4fe-477e-9b1c-e9b6ab7465b6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:28.817598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-152de819-c4fe-477e-9b1c-e9b6ab7465b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:28.940417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 684/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:28.986915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:28.987338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:28.987809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:28.989480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 682/1366] 10.222.0.52 () {54 vars in 962 bytes} [Thu Aug 4 20:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:28.996426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cc58d-728e-4873-9c86-15ac7d545d58 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:28.996707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cc58d-728e-4873-9c86-15ac7d545d58 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:28.996987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cc58d-728e-4873-9c86-15ac7d545d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:29.088604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-7c4cc58d-728e-4873-9c86-15ac7d545d58 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:29.090029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7c4cc58d-728e-4873-9c86-15ac7d545d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:29.145097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:29.281455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 685/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Aug 4 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:29.290606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:29.294183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:29.478322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3d9ed-dba7-4780-8f42-f5a64940abb3 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:29.505083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:29.614431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:29.615065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:29.615347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:29.615622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:29.618230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:29.730918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:29.733807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065745f-eea6-4625-b5b8-718add4728c6 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:29.766130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 683/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:30.814676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07365c-29a5-4fe5-9fc9-a4abfc56bd0a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:30.815234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07365c-29a5-4fe5-9fc9-a4abfc56bd0a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:30.815234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07365c-29a5-4fe5-9fc9-a4abfc56bd0a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:30.816707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 686/1369] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:47:30.822226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700dc74-545f-4f01-a87b-d67864e64065 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:30.822649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700dc74-545f-4f01-a87b-d67864e64065 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:30.822838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700dc74-545f-4f01-a87b-d67864e64065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:30.856471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:30.912269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d700dc74-545f-4f01-a87b-d67864e64065 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:30.916397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d700dc74-545f-4f01-a87b-d67864e64065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:31.036650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 684/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:31.378309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:31.379460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:31.379460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:31.380453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 687/1371] 10.222.0.52 () {54 vars in 964 bytes} [Thu Aug 4 20:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:31.386172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811d06-76ae-480c-acad-5e8d8c69ab1b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:31.386477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811d06-76ae-480c-acad-5e8d8c69ab1b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:31.386734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80811d06-76ae-480c-acad-5e8d8c69ab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:31.474317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-80811d06-76ae-480c-acad-5e8d8c69ab1b None None] MFA Rules not processed for user `49153b6e04e24c8d93d323d0a4757684`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:31.475788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-80811d06-76ae-480c-acad-5e8d8c69ab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:31.535495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:31.571449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 685/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Aug 4 20:47:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:31.579100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:31.580719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:31.712459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa37a6aa-e9e3-4af0-a38f-ed7315fd5f2f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:31.761311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:31.822894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:31.822894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:31.822894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:31.832077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:31.832077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:31.949223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:31.950632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7c717-3a41-4e6f-9e1d-00266ce88f17 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:31.973596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 688/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:32.009157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:32.009657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:32.009657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:32.011140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 686/1374] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:32.023518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736cd6-1d65-4fb9-986b-075c292eeb8c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:32.023518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736cd6-1d65-4fb9-986b-075c292eeb8c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:32.023518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa736cd6-1d65-4fb9-986b-075c292eeb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:32.096782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fa736cd6-1d65-4fb9-986b-075c292eeb8c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:32.104517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fa736cd6-1d65-4fb9-986b-075c292eeb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:32.129099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:32.211448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 689/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:32.255681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:32.257595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:32.405604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16953c-ba37-4a9d-b022-d0f0b7407859 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:32.458367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:32.526756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:32.527508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:32.527795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:32.528083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:32.529741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:32.623625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:32.625289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bcfdb-3e8a-4d3e-be62-c165186ba5a3 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:32.645140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 687/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:34.719812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:34.720362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:34.720362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:34.726535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 690/1377] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:34.727755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267413-3dd8-47b3-ba9b-1ea64d2222dc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:34.728191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267413-3dd8-47b3-ba9b-1ea64d2222dc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:34.728397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35267413-3dd8-47b3-ba9b-1ea64d2222dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:34.814442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-35267413-3dd8-47b3-ba9b-1ea64d2222dc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:34.815704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-35267413-3dd8-47b3-ba9b-1ea64d2222dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:34.823162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:34.922472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 688/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:34.968916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:34.970428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:35.143548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba466e21-0337-47af-ab9b-c179492d9323 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:35.147213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:35.280429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:35.281056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:35.281339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:35.281616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:35.283233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:35.376980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:35.378699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becd905-4f64-4810-9ac1-25933ec070f9 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:35.396986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 691/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:37.509588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:37.510164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:37.510388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:37.512049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 689/1380] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:37.518398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fee1-4fef-461d-8639-c3a89f9d550d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:37.518834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fee1-4fef-461d-8639-c3a89f9d550d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:37.519016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608fee1-4fef-461d-8639-c3a89f9d550d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:37.532299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:37.596827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4608fee1-4fef-461d-8639-c3a89f9d550d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:37.597953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4608fee1-4fef-461d-8639-c3a89f9d550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:37.716384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 692/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:37.762729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:37.764246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:37.917091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:37.930083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7202a-251b-434c-9c32-dc4fb56c4e32 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:38.063688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:38.064307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:38.064621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:38.064899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:38.066515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:38.168455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:38.169966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1124d3-942e-4487-81e9-396e452fd06c service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:38.182636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:38.192627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 690/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:40.312678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:40.313504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:40.313670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:40.315073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 693/1383] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:40.321395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51143-71d9-428c-9f85-8b5949a5d982 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:40.321909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51143-71d9-428c-9f85-8b5949a5d982 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:40.322008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f51143-71d9-428c-9f85-8b5949a5d982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:40.402761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-64f51143-71d9-428c-9f85-8b5949a5d982 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:40.404024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-64f51143-71d9-428c-9f85-8b5949a5d982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:40.511138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 691/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:40.556776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:40.558332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:40.659429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:40.738651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83387c36-2add-4995-8d3f-800eadb64c81 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:40.862721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:40.863346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:40.863631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:40.863904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:40.865667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:40.935875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:40.987452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:40.989224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426507eb-be69-4b44-b3c1-4ce2da395778 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:41.009554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 694/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:43.063698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:43.064258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:43.064387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:43.067101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 692/1386] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:43.073233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f20de-da8c-46bd-8810-a012ef79206d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:43.073845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f20de-da8c-46bd-8810-a012ef79206d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:43.073845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f20de-da8c-46bd-8810-a012ef79206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:43.146128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-851f20de-da8c-46bd-8810-a012ef79206d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:43.147877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-851f20de-da8c-46bd-8810-a012ef79206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:43.251735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 695/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:43.298286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:43.300718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:43.396762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:43.470764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de73bc5a-6bbd-48d3-b9fa-e39609aeb767 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:43.597114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:43.598011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:43.598287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:43.598558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:43.600168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:43.641266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:43.699095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:43.700757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94a196-8eeb-4e20-9879-782beeb884cd service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:43.719506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 693/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:45.805178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:45.805750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:45.805750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:45.807014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 696/1389] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:45.812424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e02e-3513-4a09-ab9b-81b265330078 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:45.812900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e02e-3513-4a09-ab9b-81b265330078 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:45.813007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e02e-3513-4a09-ab9b-81b265330078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:45.881409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c4e2e02e-3513-4a09-ab9b-81b265330078 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:45.882640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c4e2e02e-3513-4a09-ab9b-81b265330078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:45.980807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 694/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:46.024588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:46.026107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:46.058630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:46.180249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b906d13e-dfc7-409e-a126-48621b749dad None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:46.286801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:46.299374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:46.300261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:46.300580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:46.300855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:46.302488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:46.392817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:46.393809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca58d49-9922-4334-b45c-1a2169bd4d9f service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:46.413935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 697/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:47.411825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e8b9c-6715-450e-b80f-e522c06c5764 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:47.412468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e8b9c-6715-450e-b80f-e522c06c5764 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:47.412831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e8b9c-6715-450e-b80f-e522c06c5764 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:47.414782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 695/1392] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:47.420410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ede7f-76dc-494e-a5f3-bb1c1d5812d6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:47.420919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ede7f-76dc-494e-a5f3-bb1c1d5812d6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:47.421028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ede7f-76dc-494e-a5f3-bb1c1d5812d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:47.490037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-da3ede7f-76dc-494e-a5f3-bb1c1d5812d6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:47.491882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-da3ede7f-76dc-494e-a5f3-bb1c1d5812d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:47.551867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:47.595138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 698/1393] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:47.652042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:47.652381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:47.652657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:47.653995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 696/1394] 10.222.0.52 () {54 vars in 966 bytes} [Thu Aug 4 20:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:47.659028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904349d7-e6b9-4a6a-bb64-da172bb97fbf None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:47.659318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904349d7-e6b9-4a6a-bb64-da172bb97fbf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:47.659595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904349d7-e6b9-4a6a-bb64-da172bb97fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:47.736752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-904349d7-e6b9-4a6a-bb64-da172bb97fbf None None] MFA Rules not processed for user `f260187ace2043849cb3a4ab780899a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:47.737971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-904349d7-e6b9-4a6a-bb64-da172bb97fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:47.819126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 699/1395] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Aug 4 20:47:47 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:47.827567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:47.829153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:47.842787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:47.936906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed10957-760a-4f96-afa7-ec734c860ad4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:48.010511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:48.011130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:48.011409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:48.011677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:48.013324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:48.082214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:48.124885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:48.126332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1621bf18-c5b7-4531-90ea-36a6b589d7dc service glance] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:48.144653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 697/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:48.483723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:48.484292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:48.484292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:48.485827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 700/1397] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:48.490924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579aeb-567c-4547-adef-ff11b596e543 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:48.491417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579aeb-567c-4547-adef-ff11b596e543 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:48.491602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1579aeb-567c-4547-adef-ff11b596e543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:48.564130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f1579aeb-567c-4547-adef-ff11b596e543 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:48.565382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f1579aeb-567c-4547-adef-ff11b596e543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:48.669017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 698/1398] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:48.714208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:48.715711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:48.784571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:48.879425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58414df5-33b4-45e0-ad6e-5152f14793a4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:48.894975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a9b57-a674-4a99-a03d-ba44200b89bc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:48.895386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a9b57-a674-4a99-a03d-ba44200b89bc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:48.895738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a9b57-a674-4a99-a03d-ba44200b89bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:48.923541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:48.967683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a24a9b57-a674-4a99-a03d-ba44200b89bc None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:48.968937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a24a9b57-a674-4a99-a03d-ba44200b89bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:49.001996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:49.002627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:49.002910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:49.003201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:49.004931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:49.026895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:49.065465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 699/1399] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:47:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:49.100109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:49.102317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:49.110807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6101-62fc-491a-b371-f5255eec38fe None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:49.111121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6101-62fc-491a-b371-f5255eec38fe None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:49.111394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f6101-62fc-491a-b371-f5255eec38fe None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:49.112866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 700/1400] 10.222.0.52 () {54 vars in 943 bytes} [Thu Aug 4 20:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:49.119642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40204ab-fc41-4b88-bf40-1d4a8afc9789 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:49.119928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40204ab-fc41-4b88-bf40-1d4a8afc9789 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:49.120209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40204ab-fc41-4b88-bf40-1d4a8afc9789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:49.121289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 701/1401] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:47:49.206275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c40204ab-fc41-4b88-bf40-1d4a8afc9789 None None] MFA Rules not processed for user `352be9523cf1454a86c16640a87ece9c`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:49.207536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c40204ab-fc41-4b88-bf40-1d4a8afc9789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:49.271432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:49.335686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 701/1402] 10.222.0.52 () {58 vars in 1040 bytes} [Thu Aug 4 20:47:49 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:47:49.345394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:49.346608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:49.437410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea5ef54-3816-4b07-9675-58993c251fa0 service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:49.495848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:49.544732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:49.545454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:49.545738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:49.546033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:49.547952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:49.634595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:49.636205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51587738-3728-4c1b-8d75-0b687c47917f service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:49.644287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 702/1403] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:47:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:47:51.238236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:51.238751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:51.238751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:51.240382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 702/1404] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:51.246110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899111-c337-47d7-9f59-a092413ff84f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:51.246897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899111-c337-47d7-9f59-a092413ff84f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:51.247181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899111-c337-47d7-9f59-a092413ff84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:51.321240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-3e899111-c337-47d7-9f59-a092413ff84f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:51.322482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3e899111-c337-47d7-9f59-a092413ff84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:51.371694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:51.437038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 703/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:51.482782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:51.484306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:51.663472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa8b87-f31b-41f3-907e-2c888832d8c8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:51.692472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:51.805808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:51.806356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:51.806641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:51.806950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:51.808645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:51.921769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:51.923685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5695d9-81f7-40bc-9949-d2f7d6cad6e4 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:51.943745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 703/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:47:54.038608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:54.039203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:54.039203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:54.040715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 704/1407] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:47:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:47:54.046012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73390-b00f-4ff0-9a6c-cab70f743c20 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:54.046506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73390-b00f-4ff0-9a6c-cab70f743c20 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:54.046614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73390-b00f-4ff0-9a6c-cab70f743c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:54.090199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:54.118532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e0d73390-b00f-4ff0-9a6c-cab70f743c20 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:47:54.119901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e0d73390-b00f-4ff0-9a6c-cab70f743c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:54.252585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 704/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:47:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:47:54.284041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:47:54.285613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:54.460954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d05536-ff7c-4830-931d-5471cc187278 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:47:54.494104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:47:54.620278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:47:54.620943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:47:54.621233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:47:54.621509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:47:54.623196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:47:54.742522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:47:54.744330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edc97f-8b4a-4e65-8ef0-2e1bb51a7ea4 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:47:54.768522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 705/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:47:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:04.229177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=51 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:04.762062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:12.347550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec7e0-9d77-4050-95d8-cfbb2a90d0b3 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:12.348150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec7e0-9d77-4050-95d8-cfbb2a90d0b3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:12.348150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ec7e0-9d77-4050-95d8-cfbb2a90d0b3 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:12.350151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 705/1410] 10.222.0.52 () {54 vars in 918 bytes} [Thu Aug 4 20:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:12.355651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5d5e5-240b-4f08-951c-f291c5763d93 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:12.356122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5d5e5-240b-4f08-951c-f291c5763d93 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:12.356227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5d5e5-240b-4f08-951c-f291c5763d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:12.453619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-93f5d5e5-240b-4f08-951c-f291c5763d93 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:12.454859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-93f5d5e5-240b-4f08-951c-f291c5763d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:12.571051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 706/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Aug 4 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:12.593650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1db14-2d63-4d1e-b2cc-d13567dce2d3 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:12.593939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1db14-2d63-4d1e-b2cc-d13567dce2d3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:12.594220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1db14-2d63-4d1e-b2cc-d13567dce2d3 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:12.595616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 706/1412] 10.222.0.52 () {54 vars in 918 bytes} [Thu Aug 4 20:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:12.601108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5dd5-7da2-4f5d-804e-e03e73f21a0b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:12.601362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5dd5-7da2-4f5d-804e-e03e73f21a0b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:12.601662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f5dd5-7da2-4f5d-804e-e03e73f21a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:12.682721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:12.686358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c34f5dd5-7da2-4f5d-804e-e03e73f21a0b None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:12.688584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c34f5dd5-7da2-4f5d-804e-e03e73f21a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:12.816150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 707/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Thu Aug 4 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:15.308266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:15.308854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:15.308854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:15.310185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:15.315496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f958-1d47-4dbd-8a2d-4e76dc60cd87 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:15.315879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f958-1d47-4dbd-8a2d-4e76dc60cd87 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:15.316091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57f958-1d47-4dbd-8a2d-4e76dc60cd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:15.382301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2c57f958-1d47-4dbd-8a2d-4e76dc60cd87 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:15.383515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2c57f958-1d47-4dbd-8a2d-4e76dc60cd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:15.439678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:15.482094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:15.526154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:15.527541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:15.793658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858bde5-6421-4bfe-8b51-6134079d562e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:15.867654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:15.910584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:15.911200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:15.911483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:15.911785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:15.913552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:16.006658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:16.009234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b9a0bc-3b51-4015-960c-a7c354ef1722 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:16.035323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 708/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:16.936433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:16.936839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:16.936964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:16.938391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:16.943944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee40b-f9fa-4428-9a5e-a15bbd2ec0f0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:16.944523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee40b-f9fa-4428-9a5e-a15bbd2ec0f0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:16.944523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bee40b-f9fa-4428-9a5e-a15bbd2ec0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:17.019598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-94bee40b-f9fa-4428-9a5e-a15bbd2ec0f0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:17.020826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-94bee40b-f9fa-4428-9a5e-a15bbd2ec0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:17.048462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:17.122361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:48:17.277770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:17.279315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:17.431762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88be0c6b-5c36-4ad4-8ccb-9a4e65a93ae4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:17.473126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:17.552492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:17.553089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:17.553371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:17.553647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:17.555529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:17.647996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:17.649810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d8578-052b-4bfb-9eb9-a604785a271e service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:17.668758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 710/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:18.554977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:18.555459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:18.555567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:18.556910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:18.561744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34ae3c-1d84-4243-adcf-80c7ebb939fc None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:18.562152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34ae3c-1d84-4243-adcf-80c7ebb939fc None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:18.562299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34ae3c-1d84-4243-adcf-80c7ebb939fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:18.617226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:18.639878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-df34ae3c-1d84-4243-adcf-80c7ebb939fc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:18.641113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-df34ae3c-1d84-4243-adcf-80c7ebb939fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:18.741491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:48:18.899365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:18.900882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:19.048781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da9f759-ebfe-42d6-ab74-b1b6d28e67aa None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:19.061739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:19.162060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:19.162774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:19.163055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:19.163337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:19.164992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:19.255820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:19.257585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077cbfb-cd77-4381-be44-25764af53c66 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:19.275265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 711/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:20.159822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:20.160455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:20.160455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:20.162589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:20.168111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2423c6-f41f-4ab9-84bb-d97d89c5d88d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:20.168546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2423c6-f41f-4ab9-84bb-d97d89c5d88d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:20.168771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2423c6-f41f-4ab9-84bb-d97d89c5d88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:20.191339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:20.249540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1d2423c6-f41f-4ab9-84bb-d97d89c5d88d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:20.250754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1d2423c6-f41f-4ab9-84bb-d97d89c5d88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:20.345578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:20.496251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:20.497676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:20.608190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:20.643692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eaf4d6-0027-4cdd-8034-b9a7ea79e354 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:20.759275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:20.759800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:20.760080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:20.760367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:20.761990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:20.840034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:20.854368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:20.855969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a16674d-e5e6-4e87-b122-df620c26a054 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:20.874086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 713/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:21.760358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:21.760957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:21.760957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:21.762254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:21.767351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393426-21d0-4aa9-b6af-bb82fe9d6a91 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:21.767743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393426-21d0-4aa9-b6af-bb82fe9d6a91 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:21.767906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30393426-21d0-4aa9-b6af-bb82fe9d6a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:21.858018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-30393426-21d0-4aa9-b6af-bb82fe9d6a91 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:21.859739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-30393426-21d0-4aa9-b6af-bb82fe9d6a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:21.964570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:22.115259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:22.116847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:22.215188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:22.277949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ee625-c42d-49d6-87d3-89da95c2ee39 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:22.391759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:22.392327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:22.392643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:22.392949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:22.394619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:22.444002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:22.486173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:22.487768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677a98b-5ed1-41ba-940e-dae5bf10fc30 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:22.506109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 714/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:23.315921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:23.316542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:23.316542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:23.317947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:23.323352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632186-c7bf-4c03-bc25-b2ab658eb164 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:23.323791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632186-c7bf-4c03-bc25-b2ab658eb164 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:23.323900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71632186-c7bf-4c03-bc25-b2ab658eb164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:23.400266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-71632186-c7bf-4c03-bc25-b2ab658eb164 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:23.401506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-71632186-c7bf-4c03-bc25-b2ab658eb164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:23.497740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:23.649752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:23.651178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:23.700471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:23.798648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f626b5-66d6-4242-8cce-7b52abf9d862 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:23.911113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:23.911719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:23.912686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:23.912974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:23.914651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:23.921947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:24.009878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:24.011758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265aa5b-58ec-4501-bef4-379d02ed5edc service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:24.031075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 716/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:24.859622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:24.860179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:24.860179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:24.861507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:24.866697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4dec8-1b60-4f26-b6e9-6e78b4756b12 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:24.867128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4dec8-1b60-4f26-b6e9-6e78b4756b12 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:24.867243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4dec8-1b60-4f26-b6e9-6e78b4756b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:24.943383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-7fa4dec8-1b60-4f26-b6e9-6e78b4756b12 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:24.944624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7fa4dec8-1b60-4f26-b6e9-6e78b4756b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:25.011907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:25.041616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:25.191461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:25.194091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:25.213506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:25.344071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e574d6-63bd-4f19-a33d-b9fbada08fed None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:25.432530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:25.458775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:25.459324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:25.459606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:25.459879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:25.461533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:25.551000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:25.553187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6621b4f7-d2ca-4d3a-a661-b4d9769f27a6 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:25.571126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 717/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:26.372617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:26.373151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:26.373151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:26.374506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:26.379538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f77e-6cb0-42b1-a8ac-fbac8d6c0fad None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:26.379976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f77e-6cb0-42b1-a8ac-fbac8d6c0fad None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:26.380078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719f77e-6cb0-42b1-a8ac-fbac8d6c0fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:26.456298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6719f77e-6cb0-42b1-a8ac-fbac8d6c0fad None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:26.457544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6719f77e-6cb0-42b1-a8ac-fbac8d6c0fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:26.499647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:26.555907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:26.707667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:26.709355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:26.854352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d99ca-102a-4d9b-92b5-33e470f566ed None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:26.908607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:26.967161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:26.967896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:26.968179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:26.968482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:26.970083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:27.059775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:27.061391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec1a345-48ed-40ae-a076-d9c41e1f4fac service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:27.079528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 719/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:27.883897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:27.884453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:27.884690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:27.886992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:48:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:27.892253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab73-395f-4f18-a545-f8dc38731b7d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:27.892771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab73-395f-4f18-a545-f8dc38731b7d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:27.892884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12ab73-395f-4f18-a545-f8dc38731b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:27.969899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-fb12ab73-395f-4f18-a545-f8dc38731b7d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:27.971093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fb12ab73-395f-4f18-a545-f8dc38731b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:27.973137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:28.190449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:28.349990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:28.351537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:28.501230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38929ea-cdb5-4d03-b110-9026b5d54917 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:28.529432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:28.616429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:28.617049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:28.617332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:28.617606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:28.619232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:28.713482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:28.715089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ab197-d54a-4bdc-95d3-7fd7c80f3683 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:28.733501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 720/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:30.637722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:30.638333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:30.638333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:30.639643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:30.645092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdd8f-545a-4d23-9c14-9cf84c48bc93 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:30.645517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdd8f-545a-4d23-9c14-9cf84c48bc93 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:30.645707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cecdd8f-545a-4d23-9c14-9cf84c48bc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:30.689862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:30.716236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7cecdd8f-545a-4d23-9c14-9cf84c48bc93 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:30.717487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7cecdd8f-545a-4d23-9c14-9cf84c48bc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:30.814373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:30.854772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:30.856311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:30.957822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a8e02d-c72e-468e-8d00-3617502fb430 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:30.990864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:31.032048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:31.032687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:31.032968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:31.033239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:31.034843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:31.142293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:31.143637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6efaaf-0d7e-4bb6-b3a6-d8ebecbab015 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:31.162517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 722/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:33.563927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:33.564444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:33.564573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:33.565983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:33.571603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441a4f-fd6f-490b-bee5-b68cda4055a8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:33.572013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441a4f-fd6f-490b-bee5-b68cda4055a8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:33.572136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5441a4f-fd6f-490b-bee5-b68cda4055a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:33.638362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a5441a4f-fd6f-490b-bee5-b68cda4055a8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:33.639559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a5441a4f-fd6f-490b-bee5-b68cda4055a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:33.652011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:33.735607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:33.774675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:33.776297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:33.882120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2056e09-01ae-4f1b-9fd4-c386fc3e5284 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:33.894003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:33.955474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:33.956083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:33.956376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:33.956649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:33.958763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:34.069569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:34.070924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224c172-e4be-490f-9b78-8fb680fac914 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:34.088702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 723/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:36.414347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:36.414911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:36.414911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:36.416354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:36.421712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ceace-52e4-4387-a42e-5350d9c9260f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:36.422142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ceace-52e4-4387-a42e-5350d9c9260f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:36.422275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ceace-52e4-4387-a42e-5350d9c9260f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:36.471273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:36.490725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0c0ceace-52e4-4387-a42e-5350d9c9260f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:36.491949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0c0ceace-52e4-4387-a42e-5350d9c9260f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:36.588816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:36.628925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:36.630551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:36.733930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eafe29-2302-4224-92a4-a3ac4abebe00 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:36.773303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:36.808347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:36.808977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:36.809251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:36.809518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:36.811111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:36.919474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:36.920956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3599557-4615-4692-8f61-f7f49a50c380 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:36.938832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 725/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:39.103264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:39.103825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:39.103825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:39.105242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:48:39.110512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44ace-df26-4429-a24e-47dc9450285c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:39.110954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44ace-df26-4429-a24e-47dc9450285c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:39.111050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a44ace-df26-4429-a24e-47dc9450285c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:39.178800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-37a44ace-df26-4429-a24e-47dc9450285c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:39.180013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-37a44ace-df26-4429-a24e-47dc9450285c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:39.201726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:39.279135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:39.320545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:39.322066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:39.424814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6dcc0-ece1-4dc8-82d7-a9e97689e7f0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:39.446627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:39.500560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:39.501088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:39.501365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:39.501635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:39.503230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:39.614134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:39.616242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0a56c4-8657-4f6f-a6f4-5847fe4f4e41 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:39.634927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 726/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:41.781992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:41.782514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:41.782514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:41.783877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:41.790363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefcf-13cc-4fcf-9d5d-64c6f3baf83c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:41.790790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefcf-13cc-4fcf-9d5d-64c6f3baf83c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:41.790904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beddefcf-13cc-4fcf-9d5d-64c6f3baf83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:41.854129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:41.857677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-beddefcf-13cc-4fcf-9d5d-64c6f3baf83c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:41.859430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-beddefcf-13cc-4fcf-9d5d-64c6f3baf83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:41.954607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:48:41.993275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:41.994829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:42.097797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4956d-99d3-43d4-9b72-ad22b6fa4d6a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:42.146393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:42.171619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:42.172227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:42.172534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:42.172808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:42.174397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:42.283740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:42.285231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92c9f6-fb3a-4814-9013-f8e6da4dd0c0 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:42.304233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 728/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:44.452587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:44.453127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:44.453127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:44.454497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:44.459777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d09b2-093e-4491-a31d-e9dc8dcfa241 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:44.460185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d09b2-093e-4491-a31d-e9dc8dcfa241 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:44.460419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26d09b2-093e-4491-a31d-e9dc8dcfa241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:44.527096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e26d09b2-093e-4491-a31d-e9dc8dcfa241 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:44.528313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e26d09b2-093e-4491-a31d-e9dc8dcfa241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:44.557594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:44.625412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:44.665284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:44.666987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:44.774424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c3b260-0179-4ff9-a40d-aa684320f7f4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:44.805158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:44.850546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:44.851161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:44.851438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:44.851716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:44.853351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:44.964950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:44.966377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf7e807-d60f-4ab4-9bad-9d95108741f5 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:44.984529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 729/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:47.115328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:47.115892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:47.115892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:47.117327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:47.122578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6b81-adb3-4766-ba80-6799a1c906c9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:47.122967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6b81-adb3-4766-ba80-6799a1c906c9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:47.123105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f6b81-adb3-4766-ba80-6799a1c906c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:47.190876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e68f6b81-adb3-4766-ba80-6799a1c906c9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:47.193213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e68f6b81-adb3-4766-ba80-6799a1c906c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:47.201326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:47.290807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:47.330574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:47.332106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:47.436175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ece40-03f2-49e6-8edf-1bd0ff5b0330 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:47.489682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:47.507897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:47.508542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:47.508806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:47.509072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:47.510683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:47.620121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:47.621807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c04d9f-5159-4bb6-b779-6e6a74aa98b9 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:47.640513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 731/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:49.707341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:49.708503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:49.708878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:49.710348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:49.715856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec62510-3196-483a-a37e-ded34dca99f9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:49.716295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec62510-3196-483a-a37e-ded34dca99f9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:49.716467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec62510-3196-483a-a37e-ded34dca99f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:49.782661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1ec62510-3196-483a-a37e-ded34dca99f9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:49.783777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1ec62510-3196-483a-a37e-ded34dca99f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:49.823175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:49.881414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:49.918637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:49.920189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:50.024327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f9777-9565-4c21-802f-ddecfa012077 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:50.061190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:50.098628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:50.099251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:50.099535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:50.099805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:50.101499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:50.212575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:50.214035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9636f8-5f5f-45ba-ae2a-b4ef9bf58b11 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:50.232575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 732/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:52.300186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:52.300785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:52.300785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:52.302331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:52.307782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a612b-b08f-4ffd-b327-be42189fe6eb None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:52.308232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a612b-b08f-4ffd-b327-be42189fe6eb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:52.308331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a612b-b08f-4ffd-b327-be42189fe6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:52.376818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f14a612b-b08f-4ffd-b327-be42189fe6eb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:52.378036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f14a612b-b08f-4ffd-b327-be42189fe6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:52.397570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:52.475065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:52.511932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:52.513517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:52.617352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32cbdd-6551-4cd5-9c39-f7081cce1df0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:52.683708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:52.691197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:52.692426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:52.692936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:52.693315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:52.694939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:52.805512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:52.806930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c2cc9-0e30-4f75-979f-159925981d64 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:52.825146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 734/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:48:54.894269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:54.894872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:54.894872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:54.896256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:48:54.903374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ea48-d9d2-4c81-a9e5-674c1f3b48c0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:54.903885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ea48-d9d2-4c81-a9e5-674c1f3b48c0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:54.903993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29ea48-d9d2-4c81-a9e5-674c1f3b48c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:54.973910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ba29ea48-d9d2-4c81-a9e5-674c1f3b48c0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:48:54.975135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ba29ea48-d9d2-4c81-a9e5-674c1f3b48c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:55.027600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:55.080196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:48:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:48:55.117041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:48:55.118600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:55.224942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce4a7-099f-44cd-a9a8-dd3bd197c0f8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:48:55.274359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:48:55.301970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:48:55.302582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:48:55.302863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:48:55.303135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:48:55.304793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:48:55.412976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:48:55.414415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f555c4-7c0d-4eae-921e-259b1e112b9d service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:48:55.433500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 735/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:48:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:05.078374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=22 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:05.431191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=66 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:15.406618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd60f-ebb5-4e2d-9dd7-dad9f6876c95 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:15.407160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd60f-ebb5-4e2d-9dd7-dad9f6876c95 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:15.407160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffd60f-ebb5-4e2d-9dd7-dad9f6876c95 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:15.408599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 736/1471] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:49:15.414021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5b50-b5ee-4fd6-9115-bf71f525567e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:15.414479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5b50-b5ee-4fd6-9115-bf71f525567e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:15.414582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b5b50-b5ee-4fd6-9115-bf71f525567e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:15.481379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-105b5b50-b5ee-4fd6-9115-bf71f525567e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:15.482595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-105b5b50-b5ee-4fd6-9115-bf71f525567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:15.582701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 736/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:15.672801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:15.673090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:15.673372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:15.674748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 737/1473] 10.222.0.52 () {54 vars in 966 bytes} [Thu Aug 4 20:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:15.681323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897552c-10a2-45c8-b21d-55c3839a38f1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:15.681611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897552c-10a2-45c8-b21d-55c3839a38f1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:15.681887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0897552c-10a2-45c8-b21d-55c3839a38f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:15.759238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0897552c-10a2-45c8-b21d-55c3839a38f1 None None] MFA Rules not processed for user `f260187ace2043849cb3a4ab780899a9`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:15.760497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0897552c-10a2-45c8-b21d-55c3839a38f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:15.781231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:15.838671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 737/1474] 10.222.0.52 () {58 vars in 1063 bytes} [Thu Aug 4 20:49:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:15.846454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:15.849039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:15.954496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7474800-7f7c-4061-bb48-0304f03922a3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:16.030731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:16.031347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:16.031629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:16.032156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:16.033543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:16.088540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:16.145808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:16.147240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a820483-9310-4d98-bfee-2942f89c70a3 service glance] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:16.166110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 738/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:18.302674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:18.303289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:18.303289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:18.304640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 738/1476] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:18.309954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3c4f3-1705-40cf-8fe0-235456e36e12 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:18.310395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3c4f3-1705-40cf-8fe0-235456e36e12 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:18.310510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3c4f3-1705-40cf-8fe0-235456e36e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:18.375887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-caf3c4f3-1705-40cf-8fe0-235456e36e12 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:18.377390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-caf3c4f3-1705-40cf-8fe0-235456e36e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:18.474867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 739/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:49:18.497788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:18.499360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:18.628868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab770e-7f2c-4d98-9cce-a23fdeeb5c42 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:18.678191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:18.723929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:18.724574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:18.724860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:18.725134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:18.725815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:18.727566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:49:18.727566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:49:18.727566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:49:18.728732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b388142-efda-4436-b46e-3b4c4a55fdc4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:18.748353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 739/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Thu Aug 4 20:49:18 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:49:20.779952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:20.780503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:20.780503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:20.781924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 740/1479] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:20.788436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f522-df80-49c6-896d-34f6b119db5d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:20.788887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f522-df80-49c6-896d-34f6b119db5d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:20.789010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9f522-df80-49c6-896d-34f6b119db5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:20.856066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e2d9f522-df80-49c6-896d-34f6b119db5d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:20.857289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e2d9f522-df80-49c6-896d-34f6b119db5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:20.954068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 740/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:20.999126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:21.001056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:21.012405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:21.154408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89d884-f31c-490a-899d-312d26075da6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:21.236058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:21.272323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:21.272956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:21.273235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:21.273526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:21.275133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:21.366278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:21.367867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4041e-0e72-47cd-9c12-19b7c68f57b1 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:21.385975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 741/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:23.560533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:23.561080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:23.561080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:23.562464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 741/1482] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:23.568216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac413-254d-4d14-9e27-ee2d0fcf4d7d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:23.568684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac413-254d-4d14-9e27-ee2d0fcf4d7d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:23.568781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac413-254d-4d14-9e27-ee2d0fcf4d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:23.633114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f6eac413-254d-4d14-9e27-ee2d0fcf4d7d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:23.634302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f6eac413-254d-4d14-9e27-ee2d0fcf4d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:23.680905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:23.732037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 742/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:23.768376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:23.769994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:23.793944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:23.875211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d835ab-dfdb-429b-89d8-daf60366ca15 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:23.948280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:23.948928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:23.949213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:23.949490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:23.951114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:24.030939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:24.070435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:24.071787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c15ac-c8dc-4905-b95b-4e5a3263d99a service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:24.090645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 742/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:26.137650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:26.138269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:26.138269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:26.139620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 743/1485] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:26.146128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52a242-73c7-498b-af3f-8308de16c1f6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:26.146604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52a242-73c7-498b-af3f-8308de16c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:26.146712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52a242-73c7-498b-af3f-8308de16c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:26.214863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cf52a242-73c7-498b-af3f-8308de16c1f6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:26.216184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cf52a242-73c7-498b-af3f-8308de16c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:26.315505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 743/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:26.354286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:26.355809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:26.464773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223637da-7791-44f4-96b8-f04c2a722f1d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:26.536969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:26.538930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:26.539561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:26.540033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:26.540516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:26.542307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:26.650410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:26.651833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181093f2-1b2b-4025-abd9-e6b384b261a0 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:26.670569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 744/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:28.776046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:28.776651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:28.776651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:28.777912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 744/1488] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:28.783201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9368-f0f8-4d84-ba15-0faa785d2505 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:28.783630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9368-f0f8-4d84-ba15-0faa785d2505 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:28.783787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc9368-f0f8-4d84-ba15-0faa785d2505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:28.851022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-afcc9368-f0f8-4d84-ba15-0faa785d2505 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:28.852231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-afcc9368-f0f8-4d84-ba15-0faa785d2505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:28.903661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:28.949103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 745/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:28.989667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:28.991231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:29.027512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:29.094338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cdc578-0131-43b2-8c45-a4bb5fa26236 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:29.170832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:29.171365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:29.171646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:29.171921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:29.173598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:29.258880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:29.289306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:29.290659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55816e9-aa9e-4384-85b6-b7e85e524c31 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:29.308919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 745/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:31.558214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:31.558747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:31.558747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:31.560191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 746/1491] 10.222.0.52 () {54 vars in 927 bytes} [Thu Aug 4 20:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:49:31.565366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76463f-00f4-4773-bfb6-8c8fe6e28323 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:31.565809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76463f-00f4-4773-bfb6-8c8fe6e28323 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:31.565976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76463f-00f4-4773-bfb6-8c8fe6e28323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:31.631962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5a76463f-00f4-4773-bfb6-8c8fe6e28323 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:49:31.633213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5a76463f-00f4-4773-bfb6-8c8fe6e28323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:31.731357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 746/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Thu Aug 4 20:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:49:31.770248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:49:31.771802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:31.881419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16771b-b30e-4fd1-895d-a951dc3c11e3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:49:31.954971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:49:31.955586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:49:31.955869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:49:31.956179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:49:31.958661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:49:31.967638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:32.068035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:49:32.069519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e31667-7d65-4808-944b-6de713da85e7 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:49:32.087993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 747/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:49:41.729569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=82 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:49:42.085689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=50 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:48.914939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:48.915502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:48.915502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:48.993087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:50:48.994316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.092665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 747/1494] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:50:49.101440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:49.101948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:49.102054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:49.193668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:50:49.194818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.294678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 748/1495] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:50:49.300833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:49.302034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.314061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a6892-badd-4f2e-9523-f2be1557f9c9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:49.375591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:49.411961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:49.412636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:49.412923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:49.413203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:49.413920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:50:49.414231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:49.415247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:49.415247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:50:49.415247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:50:49.416138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:49.428353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 748/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:50:49.436553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:49.445723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99f8d2-7ac6-4b6a-bb0b-b6f81e2bbc0f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:49.506197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:49.541415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:49.542040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:49.542319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:49.542592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:49.543253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:49.545016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:49.545016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:50:49.545016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:50:49.546208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:49.605193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 749/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:50:49 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:50:49.609389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:49.611355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.646160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6554cf3-4cb6-41a9-97a4-f45f481a9c2a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:49.740388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:49.740984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:49.741265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:49.741543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:49.742196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:49.743301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:49.743301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:50:49.743301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:50:49.744534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:49.786846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 749/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:50:49 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:50:49.791045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:49.792580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.855144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c94092-06d2-44ed-9d16-321fde164c6a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:49.939207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:49.950043 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:49.950650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:49.950923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:49.951187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:49.951859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:49.952887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:49.952887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:50:49.952887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:50:49.954386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:49.966558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 750/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:50:49.970758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:49.972261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:49.983506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a67914-b3d4-4a4b-bf26-bbdbe4f024ad admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:50.008993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:50.080313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:50.080952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.081223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.081492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.082170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:50.083316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:50.083316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:50:50.083316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:50:50.084193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc3c3ef-1f45-4053-8730-ab228f206f0f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:50.097093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 750/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:50:50.101377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:50.102929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:50.136091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72af1dbe-d02e-4cc5-a290-c59d2901ee86 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:50.232158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:50.232915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.233196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.233468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] PATH_INFO: `/v3/projects/896cf6b7b2524fb980673d7650feff8d/users/a876acf7161a4b3bb78c2022bdd45c92/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.255193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896cf6b7b2524fb980673d7650feff8d, user_id=a876acf7161a4b3bb78c2022bdd45c92, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:50.256210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:50.256210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:50:50.256210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:50:50.258181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aa1668-dd44-44ca-89e3-76e37f9d18b8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:50.283448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 751/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:50:50 2022] PUT /identity/v3/projects/896cf6b7b2524fb980673d7650feff8d/users/a876acf7161a4b3bb78c2022bdd45c92/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:50:50.295508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415159-0b1c-466b-a9ca-20ce0da01a04 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.295820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415159-0b1c-466b-a9ca-20ce0da01a04 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.296094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415159-0b1c-466b-a9ca-20ce0da01a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.353146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6e415159-0b1c-466b-a9ca-20ce0da01a04 None None] MFA Rules not processed for user `a876acf7161a4b3bb78c2022bdd45c92`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:50:50.354339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6e415159-0b1c-466b-a9ca-20ce0da01a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:50.430737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 751/1502] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:50:50.444025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.444404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.444631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.445976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 752/1503] 10.222.0.52 () {54 vars in 964 bytes} [Thu Aug 4 20:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:50:50.450706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcc68-6bbd-4ea0-9e4f-e1ff2c116f3d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.450991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcc68-6bbd-4ea0-9e4f-e1ff2c116f3d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.451265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fcc68-6bbd-4ea0-9e4f-e1ff2c116f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.482778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:50.528115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1b9fcc68-6bbd-4ea0-9e4f-e1ff2c116f3d None None] MFA Rules not processed for user `49153b6e04e24c8d93d323d0a4757684`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:50:50.529374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1b9fcc68-6bbd-4ea0-9e4f-e1ff2c116f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:50.609725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 752/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Thu Aug 4 20:50:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:50:50.617908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:50.619449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:50.669712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:50.726109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de961-cda1-4494-a018-f13dac5aeed6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:50.801794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:50.802410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.802693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.802967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.804659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:50.901725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:50.903060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:50.906590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:50.921412 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 753/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:50:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:50:50.967644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1a21-50bc-4fff-8bba-5003ab78cc10 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:50.967935 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1a21-50bc-4fff-8bba-5003ab78cc10 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:50.968213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4c1a21-50bc-4fff-8bba-5003ab78cc10 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:50.970248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 753/1506] 10.222.0.52 () {52 vars in 830 bytes} [Thu Aug 4 20:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:50:50.973923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:50.975121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:51.074092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f055fbf-6807-4e0f-9cd6-0a5aa3bb332a service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:51.166151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:51.166813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:51.166968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:51.167237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] PATH_INFO: `/v3/users/a876acf7161a4b3bb78c2022bdd45c92` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:51.179438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a876acf7161a4b3bb78c2022bdd45c92)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:51.180486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:51.180486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:50:51.180486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:50:51.181685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:51.191234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:51.250693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a876acf7161a4b3bb78c2022bdd45c92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:51.251469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a876acf7161a4b3bb78c2022bdd45c92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:51.261730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a876acf7161a4b3bb78c2022bdd45c92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:51.276133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a876acf7161a4b3bb78c2022bdd45c92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:51.276697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a876acf7161a4b3bb78c2022bdd45c92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:51.305126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 754/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:50:50 2022] DELETE /identity/v3/users/a876acf7161a4b3bb78c2022bdd45c92 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:50:51.309792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:51.310289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:51.310396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:51.367876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:50:51.369136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:51.444311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:51.458429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 754/1508] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:50:51.467918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:51.469159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:51.568920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77534a-7d5d-4fa3-9d62-142c50c566f9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:51.622560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:50:51.645053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:51.645674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:51.645955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:51.646229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:51.647887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:51.755654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:51.757014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0676ce-33d1-4921-9926-f17cec67b68d service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:51.775071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 755/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:50:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:50:52.107080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:50:52.108489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:50:52.129451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66f53a-c3d1-4acc-af88-3a6bf20cf34e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:50:52.238653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:50:52.239273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:50:52.239548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:50:52.239820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] PATH_INFO: `/v3/projects/896cf6b7b2524fb980673d7650feff8d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:50:52.250742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=896cf6b7b2524fb980673d7650feff8d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:50:52.252075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:50:52.252075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:50:52.252075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:50:52.253198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:50:52.311451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896cf6b7b2524fb980673d7650feff8d', 'request_id': 'req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:52.322194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896cf6b7b2524fb980673d7650feff8d', 'request_id': 'req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:52.329343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896cf6b7b2524fb980673d7650feff8d', 'request_id': 'req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:52.361304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] The token cache is being invalidate because project 896cf6b7b2524fb980673d7650feff8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:50:52.361508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:52.362030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ca195c09-6a15-4b0d-8cbf-04d89cc4b20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:50:52.363365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 755/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:50:52 2022] DELETE /identity/v3/projects/896cf6b7b2524fb980673d7650feff8d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:51:01.772696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=62 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:51:02.358402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:52:23.956583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:23.957017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:23.957121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:23.958487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:23.963382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a05da-f0d6-401c-aabf-24438c5312eb None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:23.963838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a05da-f0d6-401c-aabf-24438c5312eb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:23.963943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41a05da-f0d6-401c-aabf-24438c5312eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:24.048945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e41a05da-f0d6-401c-aabf-24438c5312eb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:24.050428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e41a05da-f0d6-401c-aabf-24438c5312eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:24.154076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:24.312397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:24.314611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:24.438248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c59898f-b214-4f50-8583-4243826399a0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:24.443664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:24.444281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:24.444612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:24.444894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:24.453444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:24.455018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656666a-53d7-439d-a059-75ad546ea431 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:24.464418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 757/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:25.299896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68edf5d-c21f-4df7-bebb-663395c6ccaa None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:25.300737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68edf5d-c21f-4df7-bebb-663395c6ccaa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:25.300737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68edf5d-c21f-4df7-bebb-663395c6ccaa None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:25.302151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:25.307206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63686-9cab-41bb-b88a-5525675270af None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:25.307623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63686-9cab-41bb-b88a-5525675270af None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:25.307755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a63686-9cab-41bb-b88a-5525675270af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:25.327519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-81a63686-9cab-41bb-b88a-5525675270af None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:25.328739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-81a63686-9cab-41bb-b88a-5525675270af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:25.336555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:25.487968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68edf5d-c21f-4df7-bebb-663395c6ccaa None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:25.502763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68edf5d-c21f-4df7-bebb-663395c6ccaa None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:25.508201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:25.509202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:25.509202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:25.509423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:25.518900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:25.520629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23e376-081c-4939-90bf-a9a6f788ec91 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:25.542390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 758/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:25.776418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07414a1b-d09b-4fee-ab94-52ae2461fd2d None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:25.776709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07414a1b-d09b-4fee-ab94-52ae2461fd2d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:25.776988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07414a1b-d09b-4fee-ab94-52ae2461fd2d None None] PATH_INFO: `/v3` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:25.778310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 759/1517] 10.222.0.58 () {54 vars in 925 bytes} [Thu Aug 4 20:52:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:52:25.784449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f937-896c-4f0f-98e7-35e97a8fe9e3 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:25.784776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f937-896c-4f0f-98e7-35e97a8fe9e3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:25.785091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f937-896c-4f0f-98e7-35e97a8fe9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:25.845474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-19c1f937-896c-4f0f-98e7-35e97a8fe9e3 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:25.846671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-19c1f937-896c-4f0f-98e7-35e97a8fe9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:25.863947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:52:25.922835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 759/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Aug 4 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:35.040759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b6fd-1d8e-4cbd-8dea-7dd924118708 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:35.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b6fd-1d8e-4cbd-8dea-7dd924118708 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:35.041335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654b6fd-1d8e-4cbd-8dea-7dd924118708 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:35.042644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 760/1519] 10.222.0.52 () {54 vars in 961 bytes} [Thu Aug 4 20:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:35.048504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1053-020d-4650-a6b7-35e18a352b76 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:35.048924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1053-020d-4650-a6b7-35e18a352b76 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:35.049168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aef1053-020d-4650-a6b7-35e18a352b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:35.077211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5aef1053-020d-4650-a6b7-35e18a352b76 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:35.079258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5aef1053-020d-4650-a6b7-35e18a352b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:35.088857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 760/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Aug 4 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:35.098050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654b6fd-1d8e-4cbd-8dea-7dd924118708 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:35.112302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654b6fd-1d8e-4cbd-8dea-7dd924118708 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:35.117272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:35.117907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:35.118187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:35.118459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:35.127428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:35.129034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1409cf-fa62-4354-93fe-7a1edf99fcbf service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:35.138455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 761/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:36.901830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec983c-21b6-4781-831e-9515adf6ee06 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:36.902383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec983c-21b6-4781-831e-9515adf6ee06 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:36.902383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec983c-21b6-4781-831e-9515adf6ee06 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:36.904262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:36.909596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fda97-725f-463b-9017-085be38111c9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:36.910050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fda97-725f-463b-9017-085be38111c9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:36.910319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6fda97-725f-463b-9017-085be38111c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:36.931093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6f6fda97-725f-463b-9017-085be38111c9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:36.932314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6f6fda97-725f-463b-9017-085be38111c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:36.941916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:52:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:37.105905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec983c-21b6-4781-831e-9515adf6ee06 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:37.120304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec983c-21b6-4781-831e-9515adf6ee06 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:37.128909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:37.129427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:37.129831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:37.129985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:37.140068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:37.141715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865d000-9f81-4295-860b-d1c28d3e6063 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:37.152098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 762/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:37.999308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97f7a5-9342-4eea-a38d-839ed877d8a4 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:37.999836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97f7a5-9342-4eea-a38d-839ed877d8a4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:37.999836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97f7a5-9342-4eea-a38d-839ed877d8a4 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:38.001184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 20:52:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:38.006093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1323-aa84-46b2-89f4-953443a040ef None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:38.006545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1323-aa84-46b2-89f4-953443a040ef None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:38.006653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642a1323-aa84-46b2-89f4-953443a040ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:38.026678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-642a1323-aa84-46b2-89f4-953443a040ef None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:38.027877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-642a1323-aa84-46b2-89f4-953443a040ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:38.037084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 20:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:52:38.194075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97f7a5-9342-4eea-a38d-839ed877d8a4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:38.209265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97f7a5-9342-4eea-a38d-839ed877d8a4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:38.214730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:38.215370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:38.215652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:38.215925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:38.225601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:38.227216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaa5957-ff7b-4ff5-870c-6458e844c0ae service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:38.237158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 764/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:42.334697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:42.335499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:42.335499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:42.336682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:42.342027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e18b-368e-45d8-abd8-3ca39822efef None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:42.342500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e18b-368e-45d8-abd8-3ca39822efef None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:42.342604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36e18b-368e-45d8-abd8-3ca39822efef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:42.362377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ff36e18b-368e-45d8-abd8-3ca39822efef None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:42.363577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ff36e18b-368e-45d8-abd8-3ca39822efef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:42.371384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:42.447287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:42.449071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:42.531446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27eb996-3bc5-4656-b16b-fbd781f413ac None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:42.535028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:42.535647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:42.535933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:42.536210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:42.546568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:42.548043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f2cee-4586-417e-b7cb-8351e95eb0c0 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:42.557608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 765/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:44.557474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:44.558004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:44.558004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:44.559441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:44.564829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130dd83-dc82-49d9-897d-911d19a77326 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:44.565291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130dd83-dc82-49d9-897d-911d19a77326 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:44.565398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130dd83-dc82-49d9-897d-911d19a77326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:44.584988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a130dd83-dc82-49d9-897d-911d19a77326 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:44.586183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a130dd83-dc82-49d9-897d-911d19a77326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:44.594669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:44.630462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:44.632117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:44.645615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ec3b6-56c8-4e91-81bc-eb092515343d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:44.649245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:44.649880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:44.650188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:44.650460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:44.660143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:44.661594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9adfa9-41e0-4425-99df-7d42e0ef45bc service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:44.671279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 767/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:48.490332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:48.490910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:48.490910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:48.492762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:52:48.498478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6cf8-1007-48c1-a003-e3921d0e26d1 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:48.498937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6cf8-1007-48c1-a003-e3921d0e26d1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:48.499033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba6cf8-1007-48c1-a003-e3921d0e26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:48.518957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-24ba6cf8-1007-48c1-a003-e3921d0e26d1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:48.520147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-24ba6cf8-1007-48c1-a003-e3921d0e26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:48.527916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:52:48.829811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:48.831507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:48.897219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:52:48.912108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ff6a3-7633-441f-ab63-86957b53f88e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:48.915658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:48.916278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:48.916589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:48.916871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:48.926949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:48.928411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad07afe-3e76-42c9-87a2-f45805073359 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:48.938299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 768/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:52:50.974331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:50.974792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:50.974893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:50.976327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 20:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 20:52:50.982945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d8e-1030-4591-a78d-dd61269b6896 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:50.983435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d8e-1030-4591-a78d-dd61269b6896 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:50.983539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926d8e-1030-4591-a78d-dd61269b6896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:51.002662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8d926d8e-1030-4591-a78d-dd61269b6896 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:52:51.003848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8d926d8e-1030-4591-a78d-dd61269b6896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:51.011569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 20:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:52:51.317413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:52:51.319060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:52:51.332015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd591d23-f6ed-4e60-891b-c4c55082f2a9 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:52:51.335578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:52:51.336171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:52:51.336481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:52:51.336754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:52:51.345663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:52:51.347065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94e5a4-20b5-4c54-9bcc-fa6665e5005a service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:52:51.357093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 770/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:53:00.997899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=11 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:53:01.352126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:17.748880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:17.749539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:17.749539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:17.777425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:17.778683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:17.793820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 770/1540] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:17.802671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:17.803327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:17.803327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:17.831705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:17.832918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:17.843702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 771/1541] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:17.848423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:17.863942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef78bf-d086-47bf-b11d-036fbbc996ef None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:17.874896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:17.876174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:17.876726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:17.877215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:17.878176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:17.878712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:17.880916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:17.902267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 771/1542] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:17.911042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:17.924386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc059528-8f4f-4826-a152-9e4c88119e1e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:17.930612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:17.931260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:17.931541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:17.931815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:17.932519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:17.935197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc18680-f819-446e-9315-8ad9499e5684 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:17.994939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 772/1543] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:17 2022] POST /identity/v3/projects => generated 357 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:17.999594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.014755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489ac9b-502a-4e40-8467-916dfd8a6d1c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.082980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422030-289d-40f1-921f-b3b7fda4649a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.083189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422030-289d-40f1-921f-b3b7fda4649a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.083444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9422030-289d-40f1-921f-b3b7fda4649a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.099689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.100827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.101361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.101868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.102919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.105207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.112149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c9422030-289d-40f1-921f-b3b7fda4649a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.117453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c9422030-289d-40f1-921f-b3b7fda4649a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.135686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 773/1544] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:54:18.142611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 772/1545] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:17 2022] POST /identity/v3/users => generated 279 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:18.145011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.145298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.145572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.152922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.167381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101430dd-fea8-4406-bbcd-70ab606120a1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.174245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.181275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.181671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.182634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.183926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.184561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.185530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.188107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.206437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 773/1546] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:18.206671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 774/1547] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.210615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.212116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.230954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd45dc3-0891-40d1-9ae0-fe1668fc0f2a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.231207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d8fc-9b25-445f-aede-e5ec4d689921 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.237668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.238806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.239353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.239863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.240585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.241082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.241207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.241492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.241760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.242470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.242789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.244751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.244888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.272828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 775/1549] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:18.273219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 774/1549] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.276913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.281060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.293660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b48a0-77d3-427e-b588-bfa697879014 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.296250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb0c985-7563-4950-b03c-a2ec06ff0b34 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.313147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.314568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.314851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.315576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] PATH_INFO: `/v3/projects/c4e0a6f39aae4d88b8fc7243c9e5b6e2/users/2e106ffb64794090993851ff273bc285/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.315889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.316797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.317305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.317785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.318698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.319932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:18.319932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:54:18.319932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:54:18.321670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ec5e9-256e-4bea-8cbd-63e83927db79 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.353425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e0a6f39aae4d88b8fc7243c9e5b6e2, user_id=2e106ffb64794090993851ff273bc285, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.360364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc84b8-2a90-4e80-b071-bed382628567 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.384079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 776/1550] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:18 2022] PUT /identity/v3/projects/c4e0a6f39aae4d88b8fc7243c9e5b6e2/users/2e106ffb64794090993851ff273bc285/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:18.385497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ef65-44d6-490b-a123-c5ddeaad6a1d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.385780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ef65-44d6-490b-a123-c5ddeaad6a1d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.386051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ef65-44d6-490b-a123-c5ddeaad6a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.387761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 775/1551] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:54:18.389413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c414f-c62b-45db-bc68-472b3bcb023f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.390011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c414f-c62b-45db-bc68-472b3bcb023f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.390519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c414f-c62b-45db-bc68-472b3bcb023f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.421884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e376ef65-44d6-490b-a123-c5ddeaad6a1d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.422902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e376ef65-44d6-490b-a123-c5ddeaad6a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.442492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ff5c414f-c62b-45db-bc68-472b3bcb023f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.444139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ff5c414f-c62b-45db-bc68-472b3bcb023f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.497576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 777/1552] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.498944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.499225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.499493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.522400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.523600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.538907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 778/1553] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.538907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.540998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 776/1554] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.542284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87036-47ba-47ab-839f-4e86c577573e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.542798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87036-47ba-47ab-839f-4e86c577573e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.543186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e87036-47ba-47ab-839f-4e86c577573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.552117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc2f49-d9d3-4e39-b90b-20e89500d0b4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.557533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.558289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.558566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.558832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.559489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.560528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:18.560528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:54:18.560528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:54:18.561742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037fb6b8-65f0-4f07-9de7-f4c416ec6882 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.576105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d7e87036-47ba-47ab-839f-4e86c577573e None None] MFA Rules not processed for user `2e106ffb64794090993851ff273bc285`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.577494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d7e87036-47ba-47ab-839f-4e86c577573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.590759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 779/1555] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/users => generated 284 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:18.590759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0d3c-c4ca-4145-8c54-71d7f338f176 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.590759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0d3c-c4ca-4145-8c54-71d7f338f176 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.590759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158a0d3c-c4ca-4145-8c54-71d7f338f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.591770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:18.611288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-158a0d3c-c4ca-4145-8c54-71d7f338f176 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.612751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-158a0d3c-c4ca-4145-8c54-71d7f338f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.620734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 780/1556] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.623324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b80b90-5dec-420f-b8a7-b1e979da3375 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.623679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b80b90-5dec-420f-b8a7-b1e979da3375 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.624348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b80b90-5dec-420f-b8a7-b1e979da3375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.645726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-54b80b90-5dec-420f-b8a7-b1e979da3375 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.646923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-54b80b90-5dec-420f-b8a7-b1e979da3375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.647452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 777/1557] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.650947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4748fc5-983e-4a1c-8135-bffff037ccf2 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.650947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4748fc5-983e-4a1c-8135-bffff037ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.650947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4748fc5-983e-4a1c-8135-bffff037ccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.656251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 781/1558] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.657323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ea5d-70be-41f1-825d-a93657c6947b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.657607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ea5d-70be-41f1-825d-a93657c6947b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.657880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6ea5d-70be-41f1-825d-a93657c6947b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.677185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-39c6ea5d-70be-41f1-825d-a93657c6947b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.678404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-39c6ea5d-70be-41f1-825d-a93657c6947b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.678620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f4748fc5-983e-4a1c-8135-bffff037ccf2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.679806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f4748fc5-983e-4a1c-8135-bffff037ccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.687498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 782/1559] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.688139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 778/1560] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.689367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.689652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.689931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.692480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b602-0937-4145-949c-e74a72dd9531 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.692767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b602-0937-4145-949c-e74a72dd9531 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.693046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b602-0937-4145-949c-e74a72dd9531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.710113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.711329 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.712464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-79d0b602-0937-4145-949c-e74a72dd9531 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.713621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-79d0b602-0937-4145-949c-e74a72dd9531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.719872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 779/1561] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.723597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 783/1562] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.724838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.725122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.725395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.725973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.737165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62757d-8112-4f8a-9c56-24a97004c717 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.741773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.742431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.742712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.742983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.743690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.745097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.745326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82f8f8d-21a5-49fc-9629-003983a65c60 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.746916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.754706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 784/1563] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.755467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.755850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 780/1564] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:18.757073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.757358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.757655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.766628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b38d40-5dc7-42cc-9076-20a8f5a23916 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.771301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.771936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.772220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.772561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.773273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.775171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.779281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.780517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.788458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 781/1565] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.792189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.792815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:18.799975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6d4b1c-8e68-4357-b254-88e362cbb54d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.805198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.805808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.806090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.806327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 785/1566] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/projects => generated 355 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:18.806847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.807117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.807230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.807433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.809133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7c446-50d8-4d15-9edd-7247be52e2fe admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.828970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 782/1567] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.828970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.828970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.828970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.847448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:18.848704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:18.876610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb50d89a-9d87-4175-b900-5a277a5433e4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.881057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.881812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.882096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.882368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.883079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.883386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.884420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:18.884420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:54:18.884420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:54:18.885298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.896999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 786/1568] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.897820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.908543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a943a6-dee7-4ccb-bf5f-04d2f805523a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.913522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.914129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.914414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.914688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.915448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.915824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.917448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.922108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 783/1569] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:18.923049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.935166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 787/1570] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.935166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.940497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ff91-c417-49fc-9350-f5e877594acf None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.941423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422e698-83a0-42ef-a3cc-ff990573f790 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.945088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.945705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.945984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.946259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.946609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.946945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.947358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.947710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.948050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.948621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.948931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.949343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.951051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.958972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 784/1571] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:18.961265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.969991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 788/1572] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.969991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:18.975440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24ebf-1ccf-4713-b0b8-c437bd43cd6d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.979197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6b77-566e-463a-837e-cb4aa7a0cfa0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:18.981038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.981737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.982024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.982298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.983026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:18.983317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.984951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.985066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:18.985605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:18.986005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:18.986283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:18.986938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:18.988903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:18.998544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 785/1573] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:18.999448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.012005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d322b-60df-4279-8d78-c48d8fde7541 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.013629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 789/1574] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/users => generated 284 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.014271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.017247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.017860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.018298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.018581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.019248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.021300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.023447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e85ee-d649-403e-8f19-06a5bc829108 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.028573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.029172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.029453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.029765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.030413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.032292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.049361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 786/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:18 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.050538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.064317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 790/1576] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.065341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.079113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.116480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df371e15-120b-4761-b92e-e9c7e39ae76a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.129405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34001858-6bd0-4dc6-9ec3-e86beb4473f9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.134181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.134795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.135073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.135369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.136047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.138545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.164817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 791/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.165691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.176125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.176997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.177326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.177692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.178339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.178968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:19.179283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.180947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.194471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 787/1578] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:19.195385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.206436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981740b-3a58-45f5-8336-500a1306e36e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.239055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fc36e-06ec-44a2-8d92-ed8cf07482e0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.243959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.244761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.245044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.245348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] PATH_INFO: `/v3/projects/5c6542377aab4cf2ac6052d897fce898/users/2a88ea5faf0145ac94fb6cca314d526d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.262077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6542377aab4cf2ac6052d897fce898, user_id=2a88ea5faf0145ac94fb6cca314d526d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.264897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.265579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.266204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.266489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.266765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.267456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.269406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.282910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 792/1579] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:19 2022] PUT /identity/v3/projects/5c6542377aab4cf2ac6052d897fce898/users/2a88ea5faf0145ac94fb6cca314d526d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:19.283851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.307609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 788/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.309150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.358314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda2e988-f049-4a33-9c0c-fcb86b8333fd admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.360186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.373553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591407e7-fa39-490c-aecd-f7ca4aea523a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.378212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.378976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.379254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.379579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.380214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.381855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.392475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 789/1581] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.392853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.402906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa11f6d-ed69-40a6-a086-805cc66e4613 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.408526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.409217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.409575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.409945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.410738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.412694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.415816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.416478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.416765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.417043 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.418815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.420779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.436967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 790/1582] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.437806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.447450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520bae-dedd-4b3d-84ae-faf7c70d69d4 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.451447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 793/1583] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.452407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.453247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.453968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.454462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.454764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.455435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.457423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.479384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 791/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.479659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.480726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.491336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a13de-6c03-42ee-94cf-45fc8aeadc9d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.521693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6ef6-d9fc-4e52-bba1-5393f66162f9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.526525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.527259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.527633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.527939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.528668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.530607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cf3e97-316f-4e5c-a3cf-76ba604616d5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.556675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 794/1585] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 295 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.556675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.556675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.556675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.557394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.557940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.558983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.559503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.560792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.562788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.584182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] MFA Rules not processed for user `2a88ea5faf0145ac94fb6cca314d526d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:19.585443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:19.593440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 792/1586] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.593440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.652445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 795/1587] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:19.652978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.653343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3d2d5-985e-446d-9077-b361f5a68db7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.661318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.661952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.662239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.662515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.663181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.664633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9661220e-8b00-4dac-bc97-d873505e978d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.665180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.669367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.670037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.670372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.670656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] PATH_INFO: `/v3/projects/95e7b1a35fee47f7b3db346e012f4129/users/eae21a54753e499ca76330f8eb749ee5/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.686939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e7b1a35fee47f7b3db346e012f4129, user_id=eae21a54753e499ca76330f8eb749ee5, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.688673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 793/1588] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.689644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.692646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.704147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee477158-6dbe-41b6-bf5e-a6450218d9e2 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.704744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 796/1589] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:19 2022] PUT /identity/v3/projects/95e7b1a35fee47f7b3db346e012f4129/users/eae21a54753e499ca76330f8eb749ee5/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:19.705587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.734650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.769538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fb8a9-e754-440b-b952-6cdb47320660 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.769976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.770607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.770889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.771190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.771874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.773510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.774372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.775042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.775606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.775606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.776239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.778223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.783650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 794/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.784463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.793998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b181fc-53ab-4ee4-a955-a3d9ebd0efaa admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.798692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.799300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.799566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.799839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.800803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.804581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 797/1591] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.804581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.804830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.813312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44dafd-7683-4407-b14d-de15e76cc51e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.814800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 795/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.815700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.818052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.818687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.818976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.819301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.819992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.821636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.823991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:19.826391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b44bd9-f8e9-4084-8403-b853b7f415d0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.832148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.832889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.833267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.833520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.836637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 798/1593] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.836785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.836895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.837005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.859483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 796/1594] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:19 2022] POST /identity/v3/users => generated 287 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:19.863308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.873242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7f381-6d70-4e2b-b626-7f1ec741e917 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.878325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.879291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.879576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.879968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.881103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.882739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.892450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5508ccd-5e0b-44f6-b00b-bfe0d32b71f8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.895869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 797/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.895869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.896091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.896751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.897039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.897319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.903857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07dcd7-dd80-479c-8d36-5f12fcb2a7d2 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.909274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.909964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.910205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.910583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.911269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.912893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.923620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 798/1596] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.924452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.935031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc814a-033f-486b-9f4c-6a7ec37fdff1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.939544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.940138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.940453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.940740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.941408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.942996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.953319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 799/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.954114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.960247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.961810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.963323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63475485-5908-4f47-a2f6-7d3da31c6ca6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.968891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.969492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.969773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.970048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.970738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.971429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 799/1598] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:19.972291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.972452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.983014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f95b5c-8edb-4e5a-8248-7710df4cd102 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:19.986760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 800/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:19.986760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:19.991418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:19.992049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:19.992332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:19.992640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:19.993326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:19.996485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:19.996632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacfe60-6e08-4957-b27e-ce04fdd0b027 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.006803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.007429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.007712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.007814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 800/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.007992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.008715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.008830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.010459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.019470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825bd14-6656-4242-94d9-09759b66addb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.022077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 801/1601] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.022077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.026889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.028015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.028275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.028597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.029312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.030909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.041537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 801/1602] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.042260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db146a-5260-49a6-8ed9-52608fd3b113 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.042521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.049084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.049708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.050021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.050590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.051284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.053046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.055109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd90b90-5e32-45a3-b5ef-c54c769ce6f7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.060249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.060948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.061223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.061546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] PATH_INFO: `/v3/projects/95e7b1a35fee47f7b3db346e012f4129/users/eae21a54753e499ca76330f8eb749ee5/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.063408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95e7b1a35fee47f7b3db346e012f4129, user_id=eae21a54753e499ca76330f8eb749ee5, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.066203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.066657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 802/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.070629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.080182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 802/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/95e7b1a35fee47f7b3db346e012f4129/users/eae21a54753e499ca76330f8eb749ee5/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.081041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.082976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870bcfd-97e9-429a-8288-e01213a9c042 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.157187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.157921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.158311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.158718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] PATH_INFO: `/v3/projects/281963666cbf4245a97cbb213a3fa0e8/users/cfdbada2f1a74976b42d299a813335cf/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.166546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327902c-8885-45a1-b4d0-e8289a84448a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.172271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.173436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.173436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.173649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.174195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.176085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.176970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281963666cbf4245a97cbb213a3fa0e8, user_id=cfdbada2f1a74976b42d299a813335cf, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.178082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:20.178082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:54:20.178082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:54:20.180231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.192031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 803/1605] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.192031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.198647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 803/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/281963666cbf4245a97cbb213a3fa0e8/users/cfdbada2f1a74976b42d299a813335cf/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.199490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.206182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae537469-a185-4007-9881-937d0bc46572 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.215197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:20.216953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:20.275881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.276630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.277067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.277537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] PATH_INFO: `/v3/projects/e6ea3d73695c4b3dba67b2da3a4df789/users/87929879b1c74efea0a390a3ce1a4006/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.278673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b99276-477e-414b-9499-bd29d263cb87 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.284246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.284956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.285238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.285513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] PATH_INFO: `/v3/projects/c791c03c7d7b40fcb2e0ffad05d78172/users/a1930e444b5b47ea8e606c7ee4da600f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.295609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6ea3d73695c4b3dba67b2da3a4df789, user_id=87929879b1c74efea0a390a3ce1a4006, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.301415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.303492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c791c03c7d7b40fcb2e0ffad05d78172, user_id=a1930e444b5b47ea8e606c7ee4da600f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.306876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.318684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 804/1607] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/e6ea3d73695c4b3dba67b2da3a4df789/users/87929879b1c74efea0a390a3ce1a4006/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.322499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.327854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 804/1608] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/c791c03c7d7b40fcb2e0ffad05d78172/users/a1930e444b5b47ea8e606c7ee4da600f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.328912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.395972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc6bcf-3b8b-46da-9584-0492a37c9b91 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.400174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a66111-9d00-4179-bf75-a01c096ec7f1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.401008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.401645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.401963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.402240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.403390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.408998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5f6b9-c6d2-4bad-908d-90abe07c7b91 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.412764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.413708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.414114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.414393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] PATH_INFO: `/v3/projects/71b3821ed2f44e848959ac9d2683dab6/users/5026c90f0e3d49669c335ef542280409/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.417951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 805/1609] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:20.422648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5e51-e0f9-4e21-9299-58a37ddebd40 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.422945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5e51-e0f9-4e21-9299-58a37ddebd40 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.423227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed5e51-e0f9-4e21-9299-58a37ddebd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.433687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b3821ed2f44e848959ac9d2683dab6, user_id=5026c90f0e3d49669c335ef542280409, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.436652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.451291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 805/1610] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/71b3821ed2f44e848959ac9d2683dab6/users/5026c90f0e3d49669c335ef542280409/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.455555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.455923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e3ed5e51-e0f9-4e21-9299-58a37ddebd40 None None] MFA Rules not processed for user `eae21a54753e499ca76330f8eb749ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.457268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e3ed5e51-e0f9-4e21-9299-58a37ddebd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.539707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f96e7-3709-43f2-8f6c-68a427b24398 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.544629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.545424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.546045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.546526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] PATH_INFO: `/v3/projects/a5932111f3384861bfc778d561980e60/users/807855d862b146bdba009626f28d8d2f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.565055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 806/1611] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.565902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5932111f3384861bfc778d561980e60, user_id=807855d862b146bdba009626f28d8d2f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.566496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d206193-f395-4a48-875e-5b1e59891fde None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.566796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d206193-f395-4a48-875e-5b1e59891fde None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.567080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d206193-f395-4a48-875e-5b1e59891fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.569940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691392fe-06eb-4499-a2e8-fddc9dc66ed3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.580676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:20.591587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 806/1612] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/a5932111f3384861bfc778d561980e60/users/807855d862b146bdba009626f28d8d2f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.594686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.594686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.594686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.596370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1d206193-f395-4a48-875e-5b1e59891fde None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.597405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1d206193-f395-4a48-875e-5b1e59891fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.612406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:20.615174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.616484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.671960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 807/1613] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.676533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc10fb-4e0b-43e8-b084-fafdfc347da9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.676828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc10fb-4e0b-43e8-b084-fafdfc347da9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.677104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc10fb-4e0b-43e8-b084-fafdfc347da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.688033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 807/1614] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.689195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.701485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2674a43-8e2d-4b3e-9f55-25db9236396d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.705596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-80fc10fb-4e0b-43e8-b084-fafdfc347da9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.706539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.706851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-80fc10fb-4e0b-43e8-b084-fafdfc347da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.707396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.707773 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.708065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] PATH_INFO: `/v3/projects/1f31314d8f5b4fdcaa04a5115000baef/users/080299acab69406f918ef3fb3a5edab9/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.718667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 808/1615] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.718667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56df3-7064-4073-97a4-31a8d025b685 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.718667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56df3-7064-4073-97a4-31a8d025b685 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.718667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f56df3-7064-4073-97a4-31a8d025b685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.728204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f31314d8f5b4fdcaa04a5115000baef, user_id=080299acab69406f918ef3fb3a5edab9, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.731814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.738301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c7f56df3-7064-4073-97a4-31a8d025b685 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.739546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c7f56df3-7064-4073-97a4-31a8d025b685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.745688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 808/1616] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:20 2022] PUT /identity/v3/projects/1f31314d8f5b4fdcaa04a5115000baef/users/080299acab69406f918ef3fb3a5edab9/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:20.746651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.804474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 809/1617] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.805974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.806262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.806533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.817043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13ff39-f369-4d25-9800-7d466659662f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.820656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.821275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.821559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.821837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.825978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:20.827181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:20.835041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 810/1618] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:20.835844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.898661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.900113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.903978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c59b-39b6-4009-80ec-d28c35ba98a0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.907678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.908297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.908619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.908888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.910811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 809/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:20.911768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.920563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.920703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d247817-ef17-4d7e-b999-f0eafe4495b2 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.922136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.924844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.925719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.926007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.926268 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.934211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 811/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:20.934944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:20.937848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.939299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e028e8-f0c6-4af8-973d-ab181f975a60 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:20.943570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c8f7c2-7f36-40ec-8b63-9deeae3cb080 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:20.952741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 810/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:20.962590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:20.963556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.964125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.965453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.965938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:20.966345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:20.966692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:20.985700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:20.986584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.008438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 812/1622] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:21.013454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.016980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:21.029765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] MFA Rules not processed for user `080299acab69406f918ef3fb3a5edab9`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:21.031696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:21.036286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bacf1-0500-4245-b355-252d0dc6e637 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.048379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.050102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.051123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.052670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.070066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.072194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.089019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 813/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:21.091612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.109436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463264f2-c894-42eb-a3f6-1d23d4a77bc1 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.115738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.117765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.118067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.118353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.134750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.136965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.157860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 814/1624] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:21.159956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.166479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 811/1625] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:21.173407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.173785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77960d1-c6b6-4744-aeb6-e092951a9dd1 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.179568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.181745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.182447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.183437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.193876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:21.200185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca0f24-9323-40a2-9a77-8fd04e547d27 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.209629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.211016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.211571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.212850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.214610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.218897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.284005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 812/1626] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:21 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:21.289449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.296541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.298683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.327523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 815/1627] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:21 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:21.415757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73454fa-0a77-443d-bd45-851fa9f9fe5f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.429921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.433469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.433992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.434378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.435199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.437721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.485677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 813/1628] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:21 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:54:21.489616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.503671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ea8d7-2cff-4e0e-a73e-552ede1a22db service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.512232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.513304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.513805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.514365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.515521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.518196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.544420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 816/1629] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:54:21.547410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.559063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41da7-4196-4081-ae9e-c9a3fce89671 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.565793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.566651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.566944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.567228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] PATH_INFO: `/v3/projects/846fa8e91e804b6d935488b6bfb102c6/users/e3adf62464ca472295c99b059a77eb4f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.592254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846fa8e91e804b6d935488b6bfb102c6, user_id=e3adf62464ca472295c99b059a77eb4f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.595682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.631431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 814/1630] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:21 2022] PUT /identity/v3/projects/846fa8e91e804b6d935488b6bfb102c6/users/e3adf62464ca472295c99b059a77eb4f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:21.634209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.665226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f27ff6-3e53-4140-b981-929cc80dc85a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.784040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.785067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.785719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.786322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.788682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.791354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0eea5f-8262-4859-b753-b951197d32d7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.809049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 817/1631] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:54:21.813714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:21.830610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65689c42-8c26-4295-884e-c7fd4b4a5c42 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:21.836437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:21.838041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.838933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.840249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] PATH_INFO: `/v3/projects/846fa8e91e804b6d935488b6bfb102c6/users/e3adf62464ca472295c99b059a77eb4f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.844051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846fa8e91e804b6d935488b6bfb102c6, user_id=e3adf62464ca472295c99b059a77eb4f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:21.850439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:21.875170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 815/1632] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:21 2022] PUT /identity/v3/projects/846fa8e91e804b6d935488b6bfb102c6/users/e3adf62464ca472295c99b059a77eb4f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:21.877473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:21.878045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:21.878045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:21.912012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] MFA Rules not processed for user `e3adf62464ca472295c99b059a77eb4f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:21.914030 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:22.059529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 818/1633] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:22.073710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:22.138091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:22.202502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0099d93-3edd-4999-aa51-8953dbf36d34 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:22.210832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:22.211727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:22.212082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:22.212514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:22.225641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:22.227976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57620200-e2ff-4203-9da1-dd3c3d233946 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:22.254476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 816/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:22.421372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:22.432873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb30d0-6346-42fc-b4fa-81dd60575608 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:22.438547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:22.439276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:22.439624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:22.439843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:22.457985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:22.557436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:22.562335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64640ba8-b785-47b2-9a9b-86695c4dcc8a service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:22.573190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 819/1635] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:23.273375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad46a-ce72-49f2-9d5f-9e7a65239521 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:23.273949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad46a-ce72-49f2-9d5f-9e7a65239521 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:23.273949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edad46a-ce72-49f2-9d5f-9e7a65239521 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:23.275287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 817/1636] 10.222.0.52 () {54 vars in 962 bytes} [Thu Aug 4 20:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:23.282964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2880d9-dc21-4f75-937f-d9edfc103a5c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:23.283484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2880d9-dc21-4f75-937f-d9edfc103a5c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:23.283484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2880d9-dc21-4f75-937f-d9edfc103a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:23.341887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-3b2880d9-dc21-4f75-937f-d9edfc103a5c None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:23.344380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3b2880d9-dc21-4f75-937f-d9edfc103a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:23.486980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 820/1637] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Aug 4 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:23.488549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e280-342b-44a1-b501-9c5ebd07f8d6 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:23.488920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e280-342b-44a1-b501-9c5ebd07f8d6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:23.489348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768e280-342b-44a1-b501-9c5ebd07f8d6 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:23.491374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 818/1638] 10.222.0.52 () {54 vars in 912 bytes} [Thu Aug 4 20:54:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:23.499434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44a82-00de-467f-b24a-0b957acf049c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:23.499998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44a82-00de-467f-b24a-0b957acf049c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:23.500606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d44a82-00de-467f-b24a-0b957acf049c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:23.535697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-21d44a82-00de-467f-b24a-0b957acf049c None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:23.538662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-21d44a82-00de-467f-b24a-0b957acf049c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:23.558693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 821/1639] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:54:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:23.567580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768e280-342b-44a1-b501-9c5ebd07f8d6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:23.583054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768e280-342b-44a1-b501-9c5ebd07f8d6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:23.588095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:23.589688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:23.590365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:23.590902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:23.606665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:23.608718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cb992-ca84-4a89-aad0-b00500286017 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:23.625276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 819/1640] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:25.887434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efea4-9f78-4bea-8714-bcf314ae0f78 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:25.887993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efea4-9f78-4bea-8714-bcf314ae0f78 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:25.888245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26efea4-9f78-4bea-8714-bcf314ae0f78 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:25.889833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 822/1641] 10.222.0.52 () {54 vars in 962 bytes} [Thu Aug 4 20:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:25.898330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8a2a5-eae1-464c-bec6-d128f6ef4a50 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:25.898917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8a2a5-eae1-464c-bec6-d128f6ef4a50 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:25.899127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8a2a5-eae1-464c-bec6-d128f6ef4a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:25.941724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-61a8a2a5-eae1-464c-bec6-d128f6ef4a50 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:25.943271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-61a8a2a5-eae1-464c-bec6-d128f6ef4a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:25.957646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 820/1642] 10.222.0.52 () {58 vars in 1059 bytes} [Thu Aug 4 20:54:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:25.963552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa185ee-2d83-49e8-88d3-faa398940d50 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:25.963834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa185ee-2d83-49e8-88d3-faa398940d50 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:25.964694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa185ee-2d83-49e8-88d3-faa398940d50 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:25.969223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 823/1643] 10.222.0.52 () {54 vars in 912 bytes} [Thu Aug 4 20:54:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:25.973720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138435a2-e9cf-4780-bb05-2b20827ea014 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:25.974008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138435a2-e9cf-4780-bb05-2b20827ea014 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:25.974288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138435a2-e9cf-4780-bb05-2b20827ea014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:25.997154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-138435a2-e9cf-4780-bb05-2b20827ea014 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:25.998930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-138435a2-e9cf-4780-bb05-2b20827ea014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:26.019982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 821/1644] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:54:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:26.029720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa185ee-2d83-49e8-88d3-faa398940d50 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:26.094901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa185ee-2d83-49e8-88d3-faa398940d50 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:26.099133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:26.099759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:26.100044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:26.100433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:26.128610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:26.130140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:26.186198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 824/1645] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:31.622507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bb9f-7e98-4083-83f8-06b318204f26 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:31.623025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bb9f-7e98-4083-83f8-06b318204f26 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:31.623187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597bb9f-7e98-4083-83f8-06b318204f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:31.685214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a597bb9f-7e98-4083-83f8-06b318204f26 None None] MFA Rules not processed for user `87929879b1c74efea0a390a3ce1a4006`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:31.686482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a597bb9f-7e98-4083-83f8-06b318204f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:31.818797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 822/1646] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:31.834734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:31.855044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82360b4b-ff40-4a4a-8ccc-4e7549650cd2 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:31.867452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:31.868108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:31.868434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:31.868725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:31.887330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:31.893599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc726c5d-2359-42df-a463-73acdda1b94b service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:31.926306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 825/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:36.816313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc857ff-624f-4fa0-810a-00ba974a96a7 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:36.817455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc857ff-624f-4fa0-810a-00ba974a96a7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:36.819694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc857ff-624f-4fa0-810a-00ba974a96a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:36.848371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ccc857ff-624f-4fa0-810a-00ba974a96a7 None None] MFA Rules not processed for user `87929879b1c74efea0a390a3ce1a4006`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:36.849718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ccc857ff-624f-4fa0-810a-00ba974a96a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:36.865752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 823/1648] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:37.682830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62fa63-9b00-40b7-8eb6-e8594b05be08 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:37.683371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62fa63-9b00-40b7-8eb6-e8594b05be08 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:37.683583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62fa63-9b00-40b7-8eb6-e8594b05be08 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:37.685734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 826/1649] 10.222.0.52 () {54 vars in 912 bytes} [Thu Aug 4 20:54:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:37.693272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057e915-55ea-486d-9c57-806602f0f2aa None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:37.693764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057e915-55ea-486d-9c57-806602f0f2aa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:37.694150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057e915-55ea-486d-9c57-806602f0f2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:37.748129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f057e915-55ea-486d-9c57-806602f0f2aa None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:37.749675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f057e915-55ea-486d-9c57-806602f0f2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:37.853036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 824/1650] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:54:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:37.860181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62fa63-9b00-40b7-8eb6-e8594b05be08 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:37.876474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62fa63-9b00-40b7-8eb6-e8594b05be08 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:37.892784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:37.893832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:37.895282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:37.895732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:37.910399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:37.912398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609221c-8a7b-4d7b-ab5a-30131492c441 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:37.936232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 827/1651] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:38.959565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de794-3ebb-4e47-a5fd-a76a578ce965 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:38.960028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de794-3ebb-4e47-a5fd-a76a578ce965 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:38.960127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5de794-3ebb-4e47-a5fd-a76a578ce965 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:38.961791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 825/1652] 10.222.0.52 () {54 vars in 961 bytes} [Thu Aug 4 20:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:38.968102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d0dd-797c-4732-a7bf-eceda875234c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:38.968663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d0dd-797c-4732-a7bf-eceda875234c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:38.968663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d0dd-797c-4732-a7bf-eceda875234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:39.014458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d4e4d0dd-797c-4732-a7bf-eceda875234c None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:39.022973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d4e4d0dd-797c-4732-a7bf-eceda875234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:39.145923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:39.201189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 828/1653] 10.222.0.52 () {58 vars in 1058 bytes} [Thu Aug 4 20:54:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:39.208953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de794-3ebb-4e47-a5fd-a76a578ce965 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:39.236676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5de794-3ebb-4e47-a5fd-a76a578ce965 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:39.242909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:39.243798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:39.244228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:39.244883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:39.252197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:39.281447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:39.283371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:39.310998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 826/1654] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:39.546331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:39.546629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:39.546970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:39.634938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] MFA Rules not processed for user `a1930e444b5b47ea8e606c7ee4da600f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:39.637318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:39.849410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 829/1655] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:39.865113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:39.887035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af156bb3-095c-40ec-b9f7-b979d59271cc service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:39.915377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:39.930263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:39.930909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:39.931200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:39.931481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:39.964940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:39.966552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd2d9a6-cd53-40bf-9936-44313c852fa5 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:40.001078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 827/1656] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:40.106425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e744dc6-0443-43ef-8319-a38529815b8e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:40.125055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:40.126538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:40.127338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:40.127740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] PATH_INFO: `/v3/users/2a88ea5faf0145ac94fb6cca314d526d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:40.131496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a88ea5faf0145ac94fb6cca314d526d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:40.136894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:40.289124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a88ea5faf0145ac94fb6cca314d526d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:40.290342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a88ea5faf0145ac94fb6cca314d526d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:40.311651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a88ea5faf0145ac94fb6cca314d526d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:40.327547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a88ea5faf0145ac94fb6cca314d526d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:40.328865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a88ea5faf0145ac94fb6cca314d526d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:40.344102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:40.373597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf0db-f384-4881-8a58-ebdec7f343e4 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:40.374056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf0db-f384-4881-8a58-ebdec7f343e4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:40.374501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf0db-f384-4881-8a58-ebdec7f343e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:40.385398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 830/1657] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:54:39 2022] DELETE /identity/v3/users/2a88ea5faf0145ac94fb6cca314d526d => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:54:40.392200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:40.433124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-642bf0db-f384-4881-8a58-ebdec7f343e4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:40.434346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-642bf0db-f384-4881-8a58-ebdec7f343e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:40.647136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 828/1658] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:54:40.740703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:40.742777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:40.852070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c550-6529-49b0-bf5b-b6286ebf88f6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:40.852391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c550-6529-49b0-bf5b-b6286ebf88f6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:40.852667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c550-6529-49b0-bf5b-b6286ebf88f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:40.961148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5ad4c550-6529-49b0-bf5b-b6286ebf88f6 None None] MFA Rules not processed for user `5026c90f0e3d49669c335ef542280409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:40.962451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5ad4c550-6529-49b0-bf5b-b6286ebf88f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:41.007790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2d2e2-5965-4e7c-ab51-4017b2a9b88a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:41.051364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:41.058372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.058715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.059001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.105411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:41.109634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8cd71-f6a5-49c9-9edc-c33b5b1aaa94 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:41.147714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 831/1659] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:41.155174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89140999-2f93-40c1-884b-34642b499b9f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.155696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89140999-2f93-40c1-884b-34642b499b9f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.156129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89140999-2f93-40c1-884b-34642b499b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.241614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-89140999-2f93-40c1-884b-34642b499b9f None None] MFA Rules not processed for user `807855d862b146bdba009626f28d8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:41.246888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-89140999-2f93-40c1-884b-34642b499b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:41.260964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 829/1660] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:41.274354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.274648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.274929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.319878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] MFA Rules not processed for user `5026c90f0e3d49669c335ef542280409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:41.321183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:41.351434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 830/1661] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:41.364594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:41.366315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:41.400636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 832/1662] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:41.429630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89140999-2f93-40c1-884b-34642b499b9f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:41.431475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-89140999-2f93-40c1-884b-34642b499b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:41.605864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89140999-2f93-40c1-884b-34642b499b9f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:41.614952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:41.615598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.615884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.616230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.630549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:41.632282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fcb99f-03f6-48c3-bbe9-82b453220de7 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:41.645464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 833/1663] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:41.647889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c3161-e33f-46db-a515-0678e17ac0f0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:41.671203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:41.671858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.672144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.672658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.693052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:41.694682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95422996-358a-47fd-ae35-e3786a73c0af service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:41.708868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:41.709167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:41.709424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:41.712014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 831/1664] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:41 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:41.786427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:41.814426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] MFA Rules not processed for user `cfdbada2f1a74976b42d299a813335cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:41.817972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:42.094101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:42.094248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 834/1665] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:42.094840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:42.095324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:42.130411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] MFA Rules not processed for user `cfdbada2f1a74976b42d299a813335cf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:42.131301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:42.160766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 832/1666] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:42.453923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:42.477481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1101bd5-7249-44dd-bdda-38376a218dd9 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:42.503306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:42.506768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:42.507085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:42.507367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:42.537098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:42.541961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b05ad6-3e99-4615-bdc0-98c16250f6ac service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:42.611326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 835/1667] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:42.991820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:42.993562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:43.033182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:43.063505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e33fa4-8e08-431f-8c2f-0fd918318cf9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:43.076358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:43.088030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.088425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.089256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] PATH_INFO: `/v3/projects/5c6542377aab4cf2ac6052d897fce898` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.096586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6542377aab4cf2ac6052d897fce898)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:43.098806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:43.265312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6542377aab4cf2ac6052d897fce898', 'request_id': 'req-dcc6e7a5-e259-4efa-b67f-111d2050900a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:43.278803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6542377aab4cf2ac6052d897fce898', 'request_id': 'req-dcc6e7a5-e259-4efa-b67f-111d2050900a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:43.293259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6542377aab4cf2ac6052d897fce898', 'request_id': 'req-dcc6e7a5-e259-4efa-b67f-111d2050900a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:43.313776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] The token cache is being invalidate because project 5c6542377aab4cf2ac6052d897fce898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:54:43.314375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:43.316592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-dcc6e7a5-e259-4efa-b67f-111d2050900a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:43.322015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 833/1668] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:54:42 2022] DELETE /identity/v3/projects/5c6542377aab4cf2ac6052d897fce898 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:43.326631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.326915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.327344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.373923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:43.375193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:43.562754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 836/1669] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:43.565040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebf63-5da3-428a-9061-944bde1452e7 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.566751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebf63-5da3-428a-9061-944bde1452e7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.568207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebf63-5da3-428a-9061-944bde1452e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.670981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-226ebf63-5da3-428a-9061-944bde1452e7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:43.672445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-226ebf63-5da3-428a-9061-944bde1452e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:43.704405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 834/1670] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:43.708918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:43.724848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc86ee-5123-44a1-9511-fe483bc1077b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:43.758248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:43.759714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.760000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.760279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.767976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:43.768506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:43.770035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:43.780260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0529d8-9e37-4517-8cd5-b1e82b7f7d3f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.780841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0529d8-9e37-4517-8cd5-b1e82b7f7d3f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.784601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0529d8-9e37-4517-8cd5-b1e82b7f7d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.811630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 837/1671] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:43.818635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:43.843751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8f228-6056-4cd3-bc8c-f8e4d9174ed3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:43.862546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:43.863392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:43.863686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:43.864043 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:43.864895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:43.866878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16752b-6fe3-41bf-b42f-de287bc7f17e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:44.002406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 838/1672] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:43 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:44.003979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c5daa-cb5d-4534-8d6e-d023813fa511 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.004271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c5daa-cb5d-4534-8d6e-d023813fa511 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.004722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c5daa-cb5d-4534-8d6e-d023813fa511 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.006153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 839/1673] 10.222.0.52 () {54 vars in 912 bytes} [Thu Aug 4 20:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:44.023116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c5daa-cb5d-4534-8d6e-d023813fa511 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:44.045884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cf0529d8-9e37-4517-8cd5-b1e82b7f7d3f None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:44.055523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cf0529d8-9e37-4517-8cd5-b1e82b7f7d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:44.248484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:44.308013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 835/1674] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:54:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 528 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 04 20:54:44.315230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538db5-0a63-4428-9da9-6c57ccbde56f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.321830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538db5-0a63-4428-9da9-6c57ccbde56f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.322781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c5daa-cb5d-4534-8d6e-d023813fa511 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:44.323182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57538db5-0a63-4428-9da9-6c57ccbde56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.353551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:44.354962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.355600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.355974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.356954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:44.360955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:44.411000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 840/1675] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:44 2022] POST /identity/v3/users => generated 286 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:44.413077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:44.447783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf717d16-fc87-4452-b1ad-21a1082d266b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:44.462627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:44.463732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.465127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.465679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.466478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:44.469572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:44.489843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 841/1676] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:44.490863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-57538db5-0a63-4428-9da9-6c57ccbde56f None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:44.492133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-57538db5-0a63-4428-9da9-6c57ccbde56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:44.502560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:44.532418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95b621-313d-4638-9a34-501d4ff6f974 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:44.549133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:44.549958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.550585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.550959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.551771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:44.555515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:44.585425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 842/1677] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:44.587426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:44.661288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7644a261-adbc-4452-9688-2c4d8d7a0d07 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:44.702364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:44.705879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.709425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.712774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] PATH_INFO: `/v3/projects/adc888ab251b4357a91cfe839dcbdce8/users/e805c262d0494a769983bb3739adee9d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.765823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adc888ab251b4357a91cfe839dcbdce8, user_id=e805c262d0494a769983bb3739adee9d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:44.769803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:44.770436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 836/1678] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:44.773478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:44.773787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:44.774059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:44.813764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:44.856198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 843/1679] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:44 2022] PUT /identity/v3/projects/adc888ab251b4357a91cfe839dcbdce8/users/e805c262d0494a769983bb3739adee9d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:44.856198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:44.856198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:44.865503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] MFA Rules not processed for user `5026c90f0e3d49669c335ef542280409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:44.867369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:45.147074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 837/1680] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:44 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:45.148067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:45.153738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:45.312784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c204f12-bd97-405d-b02a-763f3014b830 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:45.336548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:45.337157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:45.337645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:45.342295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:45.394372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705bee7a-1f07-4561-ad4a-7e8593d1fe20 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:45.413159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:45.414064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:45.414511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:45.416033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:45.461448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:45.464249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671da24-97db-4dd8-aa8d-16365e672ae6 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:45.505319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 838/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:45.506673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:45.506958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:45.507233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:45.642114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:45.648902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] MFA Rules not processed for user `e805c262d0494a769983bb3739adee9d`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:45.650751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:45.653232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:45.692138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 844/1682] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:54:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 846 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:54:46.089207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 839/1683] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 583 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:46.094329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:46.146883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:46.326638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fe6605-ae62-404c-a62a-0fcab8fa5d01 service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:46.340675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:46.341333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:46.341620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:46.341895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:46.342587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:46.344562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:46.469583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 845/1684] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:46 2022] POST /identity/v3/projects => generated 373 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:46.483241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:46.499782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e898ee-e97a-4712-8288-4dde0af17b0e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:46.639828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:46.733430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:46.734167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:46.734761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:46.735277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:46.736274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:46.742562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:46.860726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 840/1685] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:46 2022] POST /identity/v3/users => generated 293 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:46.865874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:46.926764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1f8bb-76a6-4bfa-9e91-5418f173f825 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:46.956047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:46.957342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:46.957970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:46.958750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:46.960302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:46.964049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:47.017237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 846/1686] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:47.021535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:47.039680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817dbd8-c142-48e0-81f3-a88bf3252d8f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:47.067949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:47.068993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:47.069496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:47.070019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] PATH_INFO: `/v3/projects/4393b8df8730466b906b7ee11a3d7e9e/users/953648ab3c694992b0b806ab485dfae9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:47.165947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4393b8df8730466b906b7ee11a3d7e9e, user_id=953648ab3c694992b0b806ab485dfae9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:47.174388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:47.235309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:47.235446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 841/1687] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:47 2022] PUT /identity/v3/projects/4393b8df8730466b906b7ee11a3d7e9e/users/953648ab3c694992b0b806ab485dfae9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:47.249708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68216f26-80bb-4241-8eb9-d65d8c81c258 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:47.399600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:47.400972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:47.401963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:47.402590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:47.403612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:47.406005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed988069-fb18-412d-a17c-ca6969ff1cdc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:47.450892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:47.462410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 847/1688] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:47.469908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6ad48c-c854-4844-90bc-e0d4ea65b6f1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:47.478982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:47.480593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:47.481078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:47.481579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] PATH_INFO: `/v3/projects/4393b8df8730466b906b7ee11a3d7e9e/users/953648ab3c694992b0b806ab485dfae9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:47.493009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] RBAC: Authorizing `identity:create_grant(project_id=4393b8df8730466b906b7ee11a3d7e9e, user_id=953648ab3c694992b0b806ab485dfae9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:47.497688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:47.534272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 842/1689] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:47 2022] PUT /identity/v3/projects/4393b8df8730466b906b7ee11a3d7e9e/users/953648ab3c694992b0b806ab485dfae9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:47.535501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f965de-4124-4895-bbb7-fac38bfd2e9d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:47.535796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f965de-4124-4895-bbb7-fac38bfd2e9d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:47.536489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f965de-4124-4895-bbb7-fac38bfd2e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:47.660580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-80f965de-4124-4895-bbb7-fac38bfd2e9d None None] MFA Rules not processed for user `953648ab3c694992b0b806ab485dfae9`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:47.661835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-80f965de-4124-4895-bbb7-fac38bfd2e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:47.939114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 848/1690] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:47.953082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:47.954841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:48.210332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae34bb-f6ab-48dc-8bd3-4e08784edded admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:48.237218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:48.238048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:48.238367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:48.238648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:48.289525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:48.302591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:48.337520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 843/1691] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:48.577664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:48.578387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:48.578883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:48.641673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:48.643281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:48.671205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:48.819250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 849/1692] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Aug 4 20:54:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:48.878055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:48.881294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:48.911823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d129ff3-47ba-4430-9077-3d7aeca58093 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:48.943016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:48.945741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:48.951001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:48.955451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:48.958321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:48.960434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:49.006184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:49.014023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eab54d-007e-4d83-a779-13ab3ee5914a service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:49.059216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 844/1693] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:49.238823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bdb16-b97c-43ef-bc1c-d954f254ebac None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:49.251504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:49.252294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84ac1-8a31-4f05-8c8c-10ec252337cc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:49.252626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84ac1-8a31-4f05-8c8c-10ec252337cc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:49.257752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d84ac1-8a31-4f05-8c8c-10ec252337cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:49.275414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:49.275732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:49.276821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] PATH_INFO: `/v3/users/2e106ffb64794090993851ff273bc285` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:49.284458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e106ffb64794090993851ff273bc285)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:49.294990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:49.322684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e6d84ac1-8a31-4f05-8c8c-10ec252337cc None None] MFA Rules not processed for user `a1930e444b5b47ea8e606c7ee4da600f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:49.325073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e6d84ac1-8a31-4f05-8c8c-10ec252337cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:49.468564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e106ffb64794090993851ff273bc285'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:49.472999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e106ffb64794090993851ff273bc285'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:49.497816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e106ffb64794090993851ff273bc285'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:49.520590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e106ffb64794090993851ff273bc285'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:49.528208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e106ffb64794090993851ff273bc285'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:49.565173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 845/1695] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:49.565692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 850/1695] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:54:48 2022] DELETE /identity/v3/users/2e106ffb64794090993851ff273bc285 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:49.570453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:49.585044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:49.585345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:49.657544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:49.658781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:49.751452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:49.892410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:49.894391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:49.911222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 846/1696] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:49.931653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b2beb-554c-4245-b0bc-dc3561a0d739 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:49.942371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:49.943579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:49.944812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:49.946487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:49.948693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:49.952774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] PATH_INFO: `/v3/users/e805c262d0494a769983bb3739adee9d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:49.964272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e805c262d0494a769983bb3739adee9d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:49.967866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:50.076257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:50.103014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e805c262d0494a769983bb3739adee9d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:50.105674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e805c262d0494a769983bb3739adee9d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:50.130427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e805c262d0494a769983bb3739adee9d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:50.146855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e805c262d0494a769983bb3739adee9d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:50.149024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fd060590-331e-4cd3-9634-b9b8bfa64cac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e805c262d0494a769983bb3739adee9d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:50.204444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 851/1697] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:54:49 2022] DELETE /identity/v3/users/e805c262d0494a769983bb3739adee9d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:50.209917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:50.210221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:50.210504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:50.211079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8638eb7-3cbf-4a16-b069-03ffec166839 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:50.313979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:50.315628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:50.445968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:50.451336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:50.451696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:50.452007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:50.462235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:50.589988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 852/1698] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:50.633730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:50.635301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:50.656992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec27f0-08dc-441a-acce-0343c797c3d3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:50.672736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:50.673386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:50.673792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:50.674131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:50.707482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:50.709005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447105c-c837-48d4-bb50-643cf9fd7d2c service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:50.725084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:50.726634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d043f-ff18-4de9-854b-da7d399ca03f service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:50.766138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 847/1699] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 820 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:54:50.766794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 853/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:54:51.250388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862cf1-bb51-4dfd-8482-24ff57608015 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:51.251469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862cf1-bb51-4dfd-8482-24ff57608015 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:51.251922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf862cf1-bb51-4dfd-8482-24ff57608015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:51.294286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cf862cf1-bb51-4dfd-8482-24ff57608015 None None] MFA Rules not processed for user `807855d862b146bdba009626f28d8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:51.310209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cf862cf1-bb51-4dfd-8482-24ff57608015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:51.488738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:51.633920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 848/1701] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:52.475921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:52.476798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:52.477263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:52.479232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 854/1702] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:54:52.515765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:52.516595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:52.516595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:52.563974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:52.566695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:52.835178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 849/1703] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:54:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:53.156539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:53.158404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:53.224899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5908b-bd1c-4d4e-8758-4c9d1406ad32 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:53.246295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:53.249374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:53.253064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:53.253986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] PATH_INFO: `/v3/projects/c4e0a6f39aae4d88b8fc7243c9e5b6e2` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:53.257512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e0a6f39aae4d88b8fc7243c9e5b6e2)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:53.270222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: WARNING py.warnings [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:53.270222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: warnings.warn(msg) Aug 04 20:54:53.270222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: Aug 04 20:54:53.295876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:53.485419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e0a6f39aae4d88b8fc7243c9e5b6e2', 'request_id': 'req-36980503-dfd5-45f4-a609-ddcb60d88435'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:53.511766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e0a6f39aae4d88b8fc7243c9e5b6e2', 'request_id': 'req-36980503-dfd5-45f4-a609-ddcb60d88435'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:53.531227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e0a6f39aae4d88b8fc7243c9e5b6e2', 'request_id': 'req-36980503-dfd5-45f4-a609-ddcb60d88435'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:53.606434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] The token cache is being invalidate because project c4e0a6f39aae4d88b8fc7243c9e5b6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:54:53.608080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:53.609608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:53.618498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 855/1704] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:54:53 2022] DELETE /identity/v3/projects/c4e0a6f39aae4d88b8fc7243c9e5b6e2 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:53.622721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:53.656009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbb3bf-5067-4678-9723-562fcc9fb32b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:53.933513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:53.937684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:53.938152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:53.939891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] PATH_INFO: `/v3/users/eae21a54753e499ca76330f8eb749ee5` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:53.946614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae21a54753e499ca76330f8eb749ee5)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:53.953795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: WARNING py.warnings [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 20:54:53.953795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: warnings.warn(msg) Aug 04 20:54:53.953795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: Aug 04 20:54:53.958148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:54.098386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae21a54753e499ca76330f8eb749ee5'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:54.109696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae21a54753e499ca76330f8eb749ee5'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:54.111062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae21a54753e499ca76330f8eb749ee5'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:54.112686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae21a54753e499ca76330f8eb749ee5'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:54.140394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae21a54753e499ca76330f8eb749ee5'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:54.157438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:54.210256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 850/1705] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:54:53 2022] DELETE /identity/v3/users/eae21a54753e499ca76330f8eb749ee5 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:55.296493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:55.301635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:55.406344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:55.540853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36980503-dfd5-45f4-a609-ddcb60d88435 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:55.641094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:55.641862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:55.643785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:55.644185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:55.646523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:55.671004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:55.672908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:55.699366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 856/1706] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:54:56.063009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:56.064619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:56.265274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03365e-da01-467a-bc23-e3291b9fa041 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:56.272452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:56.273511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:56.274005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:56.274501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] PATH_INFO: `/v3/projects/adc888ab251b4357a91cfe839dcbdce8` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:56.277049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=adc888ab251b4357a91cfe839dcbdce8)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:56.286492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:56.423548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adc888ab251b4357a91cfe839dcbdce8', 'request_id': 'req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.437414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adc888ab251b4357a91cfe839dcbdce8', 'request_id': 'req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.447189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adc888ab251b4357a91cfe839dcbdce8', 'request_id': 'req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.480947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] The token cache is being invalidate because project adc888ab251b4357a91cfe839dcbdce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:54:56.483924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.488275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.494943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 851/1707] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:54:56 2022] DELETE /identity/v3/projects/adc888ab251b4357a91cfe839dcbdce8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:56.506285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:56.524327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc57e7f-3d21-4928-ab06-c000b05031d5 service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:56.678168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:56.683576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:56.686969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:56.687649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] PATH_INFO: `/v3/users/953648ab3c694992b0b806ab485dfae9` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:56.694790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953648ab3c694992b0b806ab485dfae9)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:56.703846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:56.771981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:56.774189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:56.809308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b6b9e2-6eaa-4980-8300-9da7ea7efe7c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:56.828481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:56.829390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:56.829866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:56.830364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] PATH_INFO: `/v3/projects/95e7b1a35fee47f7b3db346e012f4129` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:56.832403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95e7b1a35fee47f7b3db346e012f4129)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:56.834583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:56.869210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953648ab3c694992b0b806ab485dfae9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.870939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953648ab3c694992b0b806ab485dfae9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.896757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:56.909111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953648ab3c694992b0b806ab485dfae9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.936732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953648ab3c694992b0b806ab485dfae9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.937912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-980d6d06-fa75-418b-b946-8bdf38593796 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953648ab3c694992b0b806ab485dfae9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.942432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95e7b1a35fee47f7b3db346e012f4129', 'request_id': 'req-3f2f594b-3379-4fa2-b1f8-190dc4646b82'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.956624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95e7b1a35fee47f7b3db346e012f4129', 'request_id': 'req-3f2f594b-3379-4fa2-b1f8-190dc4646b82'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.965521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95e7b1a35fee47f7b3db346e012f4129', 'request_id': 'req-3f2f594b-3379-4fa2-b1f8-190dc4646b82'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.981501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] The token cache is being invalidate because project 95e7b1a35fee47f7b3db346e012f4129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:54:56.982324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.986100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 857/1708] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:54:56 2022] DELETE /identity/v3/users/953648ab3c694992b0b806ab485dfae9 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:56.987994 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3f2f594b-3379-4fa2-b1f8-190dc4646b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:56.990224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 852/1709] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:54:56 2022] DELETE /identity/v3/projects/95e7b1a35fee47f7b3db346e012f4129 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:56.995205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ef89-dada-4b59-842f-64402400506a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:56.995506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ef89-dada-4b59-842f-64402400506a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:56.995782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ef89-dada-4b59-842f-64402400506a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.022277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-aa95ef89-dada-4b59-842f-64402400506a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:57.023515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-aa95ef89-dada-4b59-842f-64402400506a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:57.138493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 858/1710] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:57.147716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.148187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.148448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.172185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:57.202482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:57.203949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:57.213100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 853/1711] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:57.217287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ef89-dada-4b59-842f-64402400506a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.231633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ef89-dada-4b59-842f-64402400506a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.244115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.245499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.247632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.247953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.248716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:57.249026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.254436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.298624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 859/1712] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:54:57.305553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.316563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b703631-4c8c-4d2f-91d2-10a1966c1234 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.324895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.325470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.325755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.326029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.326714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.328713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.375186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 854/1713] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:57 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:57.379282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.399116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b43188b-fa90-410d-af1d-4d7803ee357e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.583644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.595839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.596351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.596835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.597647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.608098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.626670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 860/1714] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:57 2022] POST /identity/v3/users => generated 285 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:57.630559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.644115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f14f035-d2d5-4a70-8126-1f481c53389e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.657938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.658763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.659051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.659330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.660142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.662784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.680311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.687692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 855/1715] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:57.694023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8c7-d794-41e1-aa63-52bd4adfcecc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.702110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.702861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.705273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.705721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.707810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.713050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.719132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.721064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:57.732100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 861/1716] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:57.737499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:57.753222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af197c9-3864-4515-8a85-257529fdb390 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.761108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.762045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.762601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.762992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b778a0-6ead-443e-a23b-b1789edd413e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:57.763333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] PATH_INFO: `/v3/projects/145b2f795a7145ea9563361084105968/users/7828549b2b1e43bebb2de2903ad6d3ff/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.773740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:57.775987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.777044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.777429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] PATH_INFO: `/v3/projects/4393b8df8730466b906b7ee11a3d7e9e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.781082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4393b8df8730466b906b7ee11a3d7e9e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.783567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145b2f795a7145ea9563361084105968, user_id=7828549b2b1e43bebb2de2903ad6d3ff, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:57.786011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.786965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645851f-48a3-4486-b92e-51d00689bd99 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:57.808780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 862/1717] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:57 2022] PUT /identity/v3/projects/145b2f795a7145ea9563361084105968/users/7828549b2b1e43bebb2de2903ad6d3ff/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:57.821418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.822163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.823984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:57.857994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:57.876480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] MFA Rules not processed for user `7828549b2b1e43bebb2de2903ad6d3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:57.882066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:57.898637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4393b8df8730466b906b7ee11a3d7e9e', 'request_id': 'req-d2254302-d522-426a-8c0f-bfa809ced352'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:57.912215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4393b8df8730466b906b7ee11a3d7e9e', 'request_id': 'req-d2254302-d522-426a-8c0f-bfa809ced352'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:57.924372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4393b8df8730466b906b7ee11a3d7e9e', 'request_id': 'req-d2254302-d522-426a-8c0f-bfa809ced352'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:57.947454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] The token cache is being invalidate because project 4393b8df8730466b906b7ee11a3d7e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:54:57.948676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:57.949951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2254302-d522-426a-8c0f-bfa809ced352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:54:57.961471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 856/1718] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:54:57 2022] DELETE /identity/v3/projects/4393b8df8730466b906b7ee11a3d7e9e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:57.964776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff673-64cf-473b-b93d-1cf470530b00 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:57.965204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff673-64cf-473b-b93d-1cf470530b00 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:57.965489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ff673-64cf-473b-b93d-1cf470530b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.019013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f95ff673-64cf-473b-b93d-1cf470530b00 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:58.020852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f95ff673-64cf-473b-b93d-1cf470530b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.048710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 863/1719] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:58.049801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.053109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.158323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 857/1720] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:58.166275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.166833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.167329 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.193320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:58.205142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba54452-8f3d-4262-a994-a119eb63a2be None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.211764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.212894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.213507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.214045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.214919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.215854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.218309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.238729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 858/1721] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:54:58.241751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.257855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df6061-c0c7-4a3c-a455-e3d741b2b4b0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.265426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.266138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.266493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.266870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.267656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:54:58.267965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.270102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.280013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 864/1722] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/projects => generated 369 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:58.281705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.290772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 859/1723] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:54:58.293310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.311202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d2ed8-15b1-43ba-8bea-8ce758488a61 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.352552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:58.384522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bb8491-b7cd-4410-92b1-fd2e1a6cf11d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.396766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.399601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.399886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.400244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.401063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.403210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.427221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.428992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.429446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.429874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.430699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.433752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.438393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 865/1724] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:58.439925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.453162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692c60da-6e03-443e-889c-e433acbc5c00 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.460062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.461706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.472573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.472573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.472573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.472573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.473156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 860/1725] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/projects => generated 365 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:54:58.477979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.490197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 866/1726] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:58.494241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.500227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:58.506756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7f7df-d953-4818-a876-dcf1e2d482fa admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.557668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837a66da-54b2-413b-ac07-86d34fbeff6e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.563092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.563730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.564007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.564287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.564988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.566942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.593118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 861/1727] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:58.593993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.594701 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.594990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.595275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] PATH_INFO: `/v3/projects/e560e615eaa74d5d9760cfd0fa243a80/users/54c6c99fb1e94b6eb9dd518b6d280489/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.597205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.613895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0cccd-2cff-4ffa-9121-b6746d95dd15 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.617362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e560e615eaa74d5d9760cfd0fa243a80, user_id=54c6c99fb1e94b6eb9dd518b6d280489, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.620362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.621055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.621249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.621334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.621608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.622307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.625470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.640857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 862/1728] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:54:58.643056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.644414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 867/1729] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:58 2022] PUT /identity/v3/projects/e560e615eaa74d5d9760cfd0fa243a80/users/54c6c99fb1e94b6eb9dd518b6d280489/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:54:58.647342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.656317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f3431-6cf5-41c0-a477-5d2b359d0690 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.727455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ea22-cb65-4b68-be08-a42178fd617a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.737134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.737836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.738256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.738542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.738811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.739244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.739590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.739940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.740282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.741021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.741111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.743011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.753894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 863/1730] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:58.756374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 868/1731] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:54:58.757631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.759059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.769886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618da7b-4aa5-46cc-b7ff-50abe86cf278 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.770770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c276ad91-c4ad-4f7a-a6a7-abeaf9d4a24f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:58.774621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.775409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.775510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:58.775856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.776116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.776288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] PATH_INFO: `/v3/projects/942ab09553774ad89fe7acbc41d14b8a/users/a570eb3ef26f4163a1cd98ed4a3d15ff/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.776614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.777061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] PATH_INFO: `/v3/projects/e560e615eaa74d5d9760cfd0fa243a80/users/54c6c99fb1e94b6eb9dd518b6d280489/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.778919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e560e615eaa74d5d9760cfd0fa243a80, user_id=54c6c99fb1e94b6eb9dd518b6d280489, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.781699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f7ef7e-2b39-4486-85e9-8d9f317ffd56 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.791717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=942ab09553774ad89fe7acbc41d14b8a, user_id=a570eb3ef26f4163a1cd98ed4a3d15ff, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:58.793100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 869/1732] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:58 2022] PUT /identity/v3/projects/e560e615eaa74d5d9760cfd0fa243a80/users/54c6c99fb1e94b6eb9dd518b6d280489/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:54:58.794711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a321b1d-8a4f-49ad-84e5-55aa85112aac admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:58.804734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.805024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.805324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.809334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 864/1733] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:58 2022] PUT /identity/v3/projects/942ab09553774ad89fe7acbc41d14b8a/users/a570eb3ef26f4163a1cd98ed4a3d15ff/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:54:58.821294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:58.821797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:58.822249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:58.830167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:58.835399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] MFA Rules not processed for user `54c6c99fb1e94b6eb9dd518b6d280489`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:58.836721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.854278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] MFA Rules not processed for user `a570eb3ef26f4163a1cd98ed4a3d15ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:58.855754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.896189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:58.906726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 870/1734] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:54:58.918163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:58.919548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:58.925655 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 865/1735] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:54:58.929927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.003993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fd7ba-5f98-4fd3-a78f-296a28d08599 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.008527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.009243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.009546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.009830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.020495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.021963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.027199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316e2ca-422e-4291-8e32-da6f57309ca5 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.032770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 871/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:54:59.034471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.035198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.035537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.035886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.042615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.044769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.091676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.093146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:54:59.102208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 866/1737] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:54:58 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:54:59.107387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.114920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b2732-8f27-486e-b943-138c4938f7df service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.224801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.225455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.225742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.226203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.262267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a20c1f3-c44f-4e32-a9d0-814a30c9338b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.273426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.274236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.274709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.275015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.275673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.277770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.318411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 867/1738] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:54:59 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:54:59.319325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.343046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36c01a-960e-4756-b244-15805bf6f4e5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.353283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.354475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.354759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.355040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.359316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.364714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.366280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:59.389662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.391583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.404972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 868/1739] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:59.409423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.412974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 872/1740] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:54:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:54:59.423434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3df558-81f8-4c01-8438-7e8552782531 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.431733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.432615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.432964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.433557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] PATH_INFO: `/v3/projects/69465a6b92e343d584d47cc427f28f10/users/3f2bd2f6b1ea4be3aec059f4bfea67b9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.464744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69465a6b92e343d584d47cc427f28f10, user_id=3f2bd2f6b1ea4be3aec059f4bfea67b9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.476799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.501253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 869/1741] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:59 2022] PUT /identity/v3/projects/69465a6b92e343d584d47cc427f28f10/users/3f2bd2f6b1ea4be3aec059f4bfea67b9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:54:59.501981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.522966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20ad453-2beb-48b5-8581-5e2857b209b5 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.654059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:54:59.689151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.690265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.690693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.691133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.692313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.701192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10885e60-78b7-491c-8074-77db5f91d781 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.735251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:54:59.742887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 873/1742] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:54:59.751447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584a16c3-0351-431e-a96d-627e03b36416 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:54:59.763986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:54:59.764715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.765002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.765353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] PATH_INFO: `/v3/projects/69465a6b92e343d584d47cc427f28f10/users/3f2bd2f6b1ea4be3aec059f4bfea67b9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.767990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=69465a6b92e343d584d47cc427f28f10, user_id=3f2bd2f6b1ea4be3aec059f4bfea67b9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:54:59.770951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:54:59.809257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 870/1743] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:54:59 2022] PUT /identity/v3/projects/69465a6b92e343d584d47cc427f28f10/users/3f2bd2f6b1ea4be3aec059f4bfea67b9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:54:59.812240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:54:59.812982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:54:59.813506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:54:59.896041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] MFA Rules not processed for user `3f2bd2f6b1ea4be3aec059f4bfea67b9`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:54:59.897608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:00.127646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 874/1744] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:54:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:00.146521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:00.263688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af719ad-a277-4412-ba46-c6b770d04bee admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:00.268946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:00.270156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:00.270596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:00.271029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:00.284678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:00.287088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37242f01-0dc0-44a2-96a5-16a665ee6f21 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:00.331090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 871/1745] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:00.835996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:00.903786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7f12-2d4b-4985-84f8-20968742b4c4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:00.909028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:00.909700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:00.910055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:00.910338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] PATH_INFO: `/v3/users/a570eb3ef26f4163a1cd98ed4a3d15ff` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:00.913161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a570eb3ef26f4163a1cd98ed4a3d15ff)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:00.916703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:00.979300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a570eb3ef26f4163a1cd98ed4a3d15ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:00.980135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a570eb3ef26f4163a1cd98ed4a3d15ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:00.988517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a570eb3ef26f4163a1cd98ed4a3d15ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.009470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a570eb3ef26f4163a1cd98ed4a3d15ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.010477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a570eb3ef26f4163a1cd98ed4a3d15ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.024757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:01.033868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 875/1746] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:00 2022] DELETE /identity/v3/users/a570eb3ef26f4163a1cd98ed4a3d15ff => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:01.038786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:01.039073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:01.039350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:01.062148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:01.063726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:01.148088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:01.149131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:01.151996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 872/1747] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:01.218231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e504b5de-7216-42da-9eff-b3a9245c9b84 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:01.221821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:01.222439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:01.222719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:01.222993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:01.233962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:01.235255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:01.252624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 876/1748] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:01.568419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:01.570864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:01.598572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfc3a30-1139-4736-b094-d8592ab2d604 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:01.604124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:01.604844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:01.605188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:01.605589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] PATH_INFO: `/v3/projects/942ab09553774ad89fe7acbc41d14b8a` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:01.606876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942ab09553774ad89fe7acbc41d14b8a)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:01.609643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:01.698351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942ab09553774ad89fe7acbc41d14b8a', 'request_id': 'req-1a443130-157b-4063-9a92-adb4f86ead26'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.707653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942ab09553774ad89fe7acbc41d14b8a', 'request_id': 'req-1a443130-157b-4063-9a92-adb4f86ead26'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.710260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:01.715525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942ab09553774ad89fe7acbc41d14b8a', 'request_id': 'req-1a443130-157b-4063-9a92-adb4f86ead26'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.732586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] The token cache is being invalidate because project 942ab09553774ad89fe7acbc41d14b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:01.733445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.733995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.736193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 873/1749] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:01 2022] DELETE /identity/v3/projects/942ab09553774ad89fe7acbc41d14b8a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:01.740150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:01.750697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf30ae9-009c-40e6-9baa-3804ee9530dc service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:01.814392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:01.815046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:01.815695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:01.815695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] PATH_INFO: `/v3/users/3f2bd2f6b1ea4be3aec059f4bfea67b9` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:01.816830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2bd2f6b1ea4be3aec059f4bfea67b9)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:01.818761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:01.864503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2bd2f6b1ea4be3aec059f4bfea67b9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.865157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2bd2f6b1ea4be3aec059f4bfea67b9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.873142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2bd2f6b1ea4be3aec059f4bfea67b9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.879819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2bd2f6b1ea4be3aec059f4bfea67b9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.880489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b1885c52-2c17-459b-a8f4-3a8fe45ed22b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2bd2f6b1ea4be3aec059f4bfea67b9'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:01.898074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 877/1750] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:01 2022] DELETE /identity/v3/users/3f2bd2f6b1ea4be3aec059f4bfea67b9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:02.165283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.167299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:02.197125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a443130-157b-4063-9a92-adb4f86ead26 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.271593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.272228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.272540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.272817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] PATH_INFO: `/v3/projects/69465a6b92e343d584d47cc427f28f10` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.274136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69465a6b92e343d584d47cc427f28f10)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.276116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.332099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69465a6b92e343d584d47cc427f28f10', 'request_id': 'req-4beae959-bc49-4a78-af38-20501f04d6e6'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:02.341523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69465a6b92e343d584d47cc427f28f10', 'request_id': 'req-4beae959-bc49-4a78-af38-20501f04d6e6'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:02.349606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69465a6b92e343d584d47cc427f28f10', 'request_id': 'req-4beae959-bc49-4a78-af38-20501f04d6e6'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:02.363741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] The token cache is being invalidate because project 69465a6b92e343d584d47cc427f28f10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:02.364141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:02.365078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4beae959-bc49-4a78-af38-20501f04d6e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:02.366857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 874/1751] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:02 2022] DELETE /identity/v3/projects/69465a6b92e343d584d47cc427f28f10 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:02.372552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.372848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.373133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.394906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:02.396086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:02.413387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:02.504406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 878/1752] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:02.512546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.512765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.513041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.533734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:02.535041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:02.543647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 875/1753] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:02.547996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.557999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472cf62b-35e0-429e-9f66-a606178095b6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.562606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.563216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.563494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.563763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.564480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:02.564782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.566358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.578156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 879/1754] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:02.582563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.592834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1fc7bc-c9c4-4c55-8bed-e17b8b046344 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.597478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.598084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.598369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.598674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.599370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.601301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.629476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 876/1755] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:02 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:02.633770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.644525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656a771-cd1e-4e58-bfe8-af8d945913e1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.701629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.702246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.702532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.702811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.703479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.705505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.726653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 880/1756] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:02 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:02.731932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.742207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ca845-1fb7-4541-ad02-c221a59b0fc5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.747566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.748264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.748580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.748860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.749562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.751185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.763220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 877/1757] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:02.767244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.778305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed2ab-23a9-49af-8eee-cef065b88072 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.782946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.783554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.783835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.784108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.784828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.786405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6aa59-fd1e-4a17-8c15-9fe3e1ac9808 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.797624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 881/1758] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:02.802023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:02.812200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016bd932-62f4-48bb-8816-4958075945a1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:02.816780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:02.817446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.817726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.818074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] PATH_INFO: `/v3/projects/f5901df4ad1643d99b5884468b91f94e/users/5d0f619c6a4f4739bc5774f075a1d65c/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.836522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5901df4ad1643d99b5884468b91f94e, user_id=5d0f619c6a4f4739bc5774f075a1d65c, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:02.839780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:02.866292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 878/1759] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:02 2022] PUT /identity/v3/projects/f5901df4ad1643d99b5884468b91f94e/users/5d0f619c6a4f4739bc5774f075a1d65c/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:02.867928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:02.868421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:02.868512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:02.905656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] MFA Rules not processed for user `5d0f619c6a4f4739bc5774f075a1d65c`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:02.907139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:02.999144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 882/1760] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:03.003612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.020571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:03.077844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861a3a-9ca2-4054-8d4b-ff2a798a58ec admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.082584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.083233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.083545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.083823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.084552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.086586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.118709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 879/1761] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:03 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:03.120786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.131787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d5e3d-8dd9-4ab4-bad6-370171feba53 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.137222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:03.192820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.194100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.194753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.195377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.196736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.201109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.235737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 883/1762] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:03 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:03.240400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.255923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfec4c-4117-4353-ab15-20df22e7424c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.263095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.263801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.264159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.264544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.265331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.266968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.278772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 880/1763] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:03.283031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.296932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de7a02-11f2-4e36-aec2-127c51a8eb08 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.302544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.303217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.303506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.303785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] PATH_INFO: `/v3/projects/19447f679aa942bd83e986e0649c1403/users/0de213483ca54b54810ed4f20a6675a7/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.321201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19447f679aa942bd83e986e0649c1403, user_id=0de213483ca54b54810ed4f20a6675a7, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.324852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.339325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 884/1764] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:03 2022] PUT /identity/v3/projects/19447f679aa942bd83e986e0649c1403/users/0de213483ca54b54810ed4f20a6675a7/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:03.343155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.356612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.358266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c126df4-6749-4b2a-89b5-5ea1a0358b2c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.358951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:03.447283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.449007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.449288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.449562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.450267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.451877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.453259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9401089-2e48-4b58-91e3-2b68853d0fc0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.460072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.460849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.461206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.461572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] PATH_INFO: `/v3/users/7828549b2b1e43bebb2de2903ad6d3ff` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.462895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7828549b2b1e43bebb2de2903ad6d3ff)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.465094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.467097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 881/1765] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:03 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:03.471718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.490459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2419b1-95db-4c45-84a6-21d11e2e445d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.504257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.504257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.504257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.504257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] PATH_INFO: `/v3/projects/19447f679aa942bd83e986e0649c1403/users/0de213483ca54b54810ed4f20a6675a7/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.506148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19447f679aa942bd83e986e0649c1403, user_id=0de213483ca54b54810ed4f20a6675a7, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:03.518262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848e848-c723-4c2e-b2c1-43b32418b269 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:03.556444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 882/1766] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:03 2022] PUT /identity/v3/projects/19447f679aa942bd83e986e0649c1403/users/0de213483ca54b54810ed4f20a6675a7/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:03.565268 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.567045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.567565 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.582627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7828549b2b1e43bebb2de2903ad6d3ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:03.583557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7828549b2b1e43bebb2de2903ad6d3ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:03.613636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] MFA Rules not processed for user `0de213483ca54b54810ed4f20a6675a7`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:03.615717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:03.617027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7828549b2b1e43bebb2de2903ad6d3ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:03.639145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7828549b2b1e43bebb2de2903ad6d3ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:03.640257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ed38cf52-ef38-4a79-b3f4-8cbcb840bd39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7828549b2b1e43bebb2de2903ad6d3ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:03.662886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:03.706499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 885/1767] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:03 2022] DELETE /identity/v3/users/7828549b2b1e43bebb2de2903ad6d3ff => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:03.719972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.720276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.720646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:03.740401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:03.757442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 883/1768] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:03.767431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.769422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:03.771799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:03.773082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:03.871587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec6dd4-c394-4278-9676-d4883714084f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:03.946433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 886/1769] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:03.960911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:03.962499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:03.990635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:03.991279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:03.991570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:03.991848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:04.057428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706be46-8a2e-4c0f-9ab9-c40002b48cf3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:04.061877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:04.063199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:04.063480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:04.063952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:04.078126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:04.079595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:04.085738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:04.088759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:04.098132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 887/1770] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:04.115000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 884/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:04.258493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:04.273489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb52cf9-266f-41fc-88b8-3dbb7c05dd39 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:04.280797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:04.292999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:04.293367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:04.293652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:04.296476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:04.422063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:04.425050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:04.441322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 888/1772] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:04.672063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:04.673601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:04.700823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebedb46-6426-4c20-a5c5-5f391dc9a295 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:04.717297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:04.718625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:04.718928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:04.719361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] PATH_INFO: `/v3/projects/145b2f795a7145ea9563361084105968` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:04.721199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=145b2f795a7145ea9563361084105968)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:04.724650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:04.809341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145b2f795a7145ea9563361084105968', 'request_id': 'req-44b875a5-17e6-4f7f-bbf4-5899aca48afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:04.813993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:04.819634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145b2f795a7145ea9563361084105968', 'request_id': 'req-44b875a5-17e6-4f7f-bbf4-5899aca48afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:04.827542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145b2f795a7145ea9563361084105968', 'request_id': 'req-44b875a5-17e6-4f7f-bbf4-5899aca48afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:04.842262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] The token cache is being invalidate because project 145b2f795a7145ea9563361084105968 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:04.842562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:04.843229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:04.844795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 885/1773] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:04 2022] DELETE /identity/v3/projects/145b2f795a7145ea9563361084105968 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:04.848870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:04.865226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1876e-2e4c-474f-840f-bd9bdb61781d service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:04.917923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:04.969890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:04.970815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:04.971263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:04.971675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] PATH_INFO: `/v3/users/54c6c99fb1e94b6eb9dd518b6d280489` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:04.973211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c6c99fb1e94b6eb9dd518b6d280489)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:04.976053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:05.051765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c6c99fb1e94b6eb9dd518b6d280489'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.052576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c6c99fb1e94b6eb9dd518b6d280489'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.062404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c6c99fb1e94b6eb9dd518b6d280489'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.070683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c6c99fb1e94b6eb9dd518b6d280489'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.071504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cc1969b2-40e9-4455-9efc-1352831e0f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c6c99fb1e94b6eb9dd518b6d280489'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.103855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 889/1774] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:04 2022] DELETE /identity/v3/users/54c6c99fb1e94b6eb9dd518b6d280489 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:05.410804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:05.412403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:05.428200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b875a5-17e6-4f7f-bbf4-5899aca48afd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:05.495651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:05.497003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:05.497153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:05.497531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] PATH_INFO: `/v3/projects/e560e615eaa74d5d9760cfd0fa243a80` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:05.499629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] RBAC: Authorizing `identity:delete_project(project_id=e560e615eaa74d5d9760cfd0fa243a80)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:05.503323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:05.561300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e560e615eaa74d5d9760cfd0fa243a80', 'request_id': 'req-2cf0b38a-c559-4541-a135-585ecb1259da'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.569853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e560e615eaa74d5d9760cfd0fa243a80', 'request_id': 'req-2cf0b38a-c559-4541-a135-585ecb1259da'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.577895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e560e615eaa74d5d9760cfd0fa243a80', 'request_id': 'req-2cf0b38a-c559-4541-a135-585ecb1259da'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.590768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] The token cache is being invalidate because project e560e615eaa74d5d9760cfd0fa243a80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:05.591139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.591877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2cf0b38a-c559-4541-a135-585ecb1259da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:05.593494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 886/1775] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:05 2022] DELETE /identity/v3/projects/e560e615eaa74d5d9760cfd0fa243a80 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:05.598606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:05.598899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:05.599185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:05.622828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:05.624082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:05.687999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 890/1776] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:05.695752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:05.696254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:05.696383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:05.720149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:05.721678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:05.738213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:05.745103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 887/1777] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:05.751114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4af6be-a768-44b8-a401-b10f88c7f6ca None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:05.755980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:05.756665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:05.756984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:05.757298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:05.758000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:05.758308 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:05.760043 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:05.775404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 891/1778] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:05.780731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:05.790670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfb4db-9103-49bd-9d21-85b89c298ce4 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:05.799271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:05.799943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:05.801088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:05.801364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:05.802088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:05.805915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:05.858485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 888/1779] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:05 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:05.860560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:05.890858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef15554-d7f8-41d7-867c-6d6b4d05d2e2 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:05.969692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:06.017295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.020639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.021578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.023309 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.026091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.036095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.098030 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 892/1780] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:05 2022] POST /identity/v3/users => generated 294 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:06.099799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.114151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161e948-512b-404a-8ae8-b880eccab81b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.129584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.130251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.130487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.130762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.134337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.134337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.164702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 889/1781] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:06.165616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.184473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd01144-712b-4fb1-aa93-55747a12d89f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.200388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.201232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.201548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.201922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.202719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.204611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173ce7a-3a2a-4c88-987f-bbd38436bc74 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.240725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 893/1782] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:06.245972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.261883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ee876-8879-4650-8967-025b94588b5a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.280666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.281413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.281697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.281979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] PATH_INFO: `/v3/projects/e6e7382a893f40a0bc8771b419e02b1f/users/be19a9cf878241dcb8bfea16086de7ff/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.321146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e7382a893f40a0bc8771b419e02b1f, user_id=be19a9cf878241dcb8bfea16086de7ff, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.325028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.345481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 890/1783] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:06 2022] PUT /identity/v3/projects/e6e7382a893f40a0bc8771b419e02b1f/users/be19a9cf878241dcb8bfea16086de7ff/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:06.358164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.358634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.359563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.401352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] MFA Rules not processed for user `be19a9cf878241dcb8bfea16086de7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:06.402924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:06.568180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 894/1784] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:06.573740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.593308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:06.672423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb760f-5876-4946-853c-dc8f144b63a9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.678502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.679245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.679521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.679804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.681556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.683527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.718016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 891/1785] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:06 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:06.723098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.748457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b439fba-3ac0-41dd-938e-f66a7c884894 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.828609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.829361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.829706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.830049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.831350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.833131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.860980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 895/1786] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:06 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:06.866533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.878423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450477a-e419-4626-a7ec-b0b2fefbb0fd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.884498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.886300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.886519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.886659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:06.887340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:06.889001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:06.901853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 892/1787] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:06.906640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:06.939469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c519e5-8b84-477c-ab57-dd0b01778c6e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:06.952049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:06.959817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:06.960143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:06.960406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] PATH_INFO: `/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8/users/7b9c5c3ef19d4bc6bbaa16907cb22d82/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.002659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1287b0be3ab443c48ee03a35dcafc3c8, user_id=7b9c5c3ef19d4bc6bbaa16907cb22d82, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.005678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.031844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 896/1788] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:06 2022] PUT /identity/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8/users/7b9c5c3ef19d4bc6bbaa16907cb22d82/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:07.036511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.053820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374806e6-e903-4cd4-a437-6cd0439c3a14 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.191443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.192218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.192801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.193149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.194664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.200382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbd69b6-a0e1-40dd-99cc-fad14e4da401 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.245428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 893/1789] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:07.245608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.283092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4737d4-3c4c-4399-b636-36b1e90f367d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.296319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.299100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.299585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.300027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] PATH_INFO: `/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8/users/7b9c5c3ef19d4bc6bbaa16907cb22d82/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.305286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1287b0be3ab443c48ee03a35dcafc3c8, user_id=7b9c5c3ef19d4bc6bbaa16907cb22d82, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.314821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.348730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 897/1790] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:07 2022] PUT /identity/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8/users/7b9c5c3ef19d4bc6bbaa16907cb22d82/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:07.357264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a671cd-3d02-42d9-a933-1b4819cc79ab None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.357756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a671cd-3d02-42d9-a933-1b4819cc79ab None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.358176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a671cd-3d02-42d9-a933-1b4819cc79ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.424500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b8a671cd-3d02-42d9-a933-1b4819cc79ab None None] MFA Rules not processed for user `7b9c5c3ef19d4bc6bbaa16907cb22d82`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:07.426414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b8a671cd-3d02-42d9-a933-1b4819cc79ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:07.564642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 894/1791] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:07.575397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.577857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:07.588539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a5c-ab20-45d1-b41b-b66ba9e8c50b None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.588753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a5c-ab20-45d1-b41b-b66ba9e8c50b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.589121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b53a5c-ab20-45d1-b41b-b66ba9e8c50b None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.591487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 895/1792] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:55:07.593970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:07.607361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.607592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.607873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.634562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:07.668973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:07.670775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:07.727608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf82d4e-f8c7-4168-b3d0-4e9c6e582fd4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.733551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.734208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.734560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.735062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.748201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.750334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.770782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 898/1793] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:07.790415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 896/1794] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:07.804239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.810212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:07.827223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.837762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:07.840097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fa2f8-b6f5-4382-8dfb-d22d5cea0477 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.847160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.847699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.847999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.849179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.860027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.861942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.880750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 899/1795] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:07.881600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:07.930580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021ac70-1e0e-4f75-91fd-957834943f72 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.932717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.933462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.933810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.934465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] PATH_INFO: `/v3/users/5d0f619c6a4f4739bc5774f075a1d65c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:07.936447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0f619c6a4f4739bc5774f075a1d65c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:07.938732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:07.990731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d6ff0-aa3d-4197-84cb-4b335416d32d service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:07.996995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:07.997995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:07.998533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:07.999391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] PATH_INFO: `/v3/users/be19a9cf878241dcb8bfea16086de7ff` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.000896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be19a9cf878241dcb8bfea16086de7ff)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:08.003248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:08.003507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0f619c6a4f4739bc5774f075a1d65c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.016967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0f619c6a4f4739bc5774f075a1d65c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.018493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0f619c6a4f4739bc5774f075a1d65c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.019132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0f619c6a4f4739bc5774f075a1d65c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.038977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74ee3313-e7c4-4705-a7ca-96522d730481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0f619c6a4f4739bc5774f075a1d65c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.056921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 897/1796] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:07 2022] DELETE /identity/v3/users/5d0f619c6a4f4739bc5774f075a1d65c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:08.058990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:08.059221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:08.059511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.071209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be19a9cf878241dcb8bfea16086de7ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.072114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be19a9cf878241dcb8bfea16086de7ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.085542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be19a9cf878241dcb8bfea16086de7ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.091507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:08.093376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.099791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be19a9cf878241dcb8bfea16086de7ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.100766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-db33e8e1-d704-4b48-9745-314ba136fc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be19a9cf878241dcb8bfea16086de7ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.106981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:08.148399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 900/1797] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:07 2022] DELETE /identity/v3/users/be19a9cf878241dcb8bfea16086de7ff => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:08.153533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:08.154129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:08.154625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.182203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:08.183692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.195396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:08.200587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 898/1798] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:08.264583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:08.266175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.304775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 901/1799] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:08.320393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:08.320393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.361796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d179e-64fb-4faa-adf5-c0ac4626b77b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:08.390217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c6c50-f6d6-4ffa-ad80-0b6a10b96005 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:08.394237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:08.394872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:08.395157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:08.395492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.405985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:08.407433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:08.418451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 902/1800] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:08.429050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:08.429685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:08.429976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:08.430255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.445519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:08.448014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:08.458352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 899/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:08.814645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:08.816182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.830826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c0708-ff66-4436-92e4-8a6e259aba1b service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:08.837126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:08.837835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:08.838832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:08.839098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] PATH_INFO: `/v3/projects/e6e7382a893f40a0bc8771b419e02b1f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:08.840742 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e7382a893f40a0bc8771b419e02b1f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:08.842978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:08.905704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e7382a893f40a0bc8771b419e02b1f', 'request_id': 'req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.912855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e7382a893f40a0bc8771b419e02b1f', 'request_id': 'req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.917754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:08.919214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.922451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e7382a893f40a0bc8771b419e02b1f', 'request_id': 'req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.934608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] The token cache is being invalidate because project e6e7382a893f40a0bc8771b419e02b1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:08.934974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.935768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:08.938103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 903/1802] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:08 2022] DELETE /identity/v3/projects/e6e7382a893f40a0bc8771b419e02b1f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:08.942280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:08.943477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:08.949124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c7a21-c32d-4c80-813f-937f8e42cf12 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:08.988218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:09.010666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:09.011354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.011638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.011918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] PATH_INFO: `/v3/projects/f5901df4ad1643d99b5884468b91f94e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.012399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9902ce-52f7-497d-a5c2-4d15f562d1fd admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:09.013111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5901df4ad1643d99b5884468b91f94e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:09.015074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:09.017197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:09.017872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.018218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.018642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] PATH_INFO: `/v3/users/7b9c5c3ef19d4bc6bbaa16907cb22d82` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.019833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b9c5c3ef19d4bc6bbaa16907cb22d82)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:09.021789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:09.032052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:09.064296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5901df4ad1643d99b5884468b91f94e', 'request_id': 'req-613d6311-4e7c-408b-8f23-a4d0d55d2b34'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.070118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b9c5c3ef19d4bc6bbaa16907cb22d82'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.071040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b9c5c3ef19d4bc6bbaa16907cb22d82'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.074463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5901df4ad1643d99b5884468b91f94e', 'request_id': 'req-613d6311-4e7c-408b-8f23-a4d0d55d2b34'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.079622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b9c5c3ef19d4bc6bbaa16907cb22d82'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.081130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5901df4ad1643d99b5884468b91f94e', 'request_id': 'req-613d6311-4e7c-408b-8f23-a4d0d55d2b34'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.087070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b9c5c3ef19d4bc6bbaa16907cb22d82'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.087739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b9c5c3ef19d4bc6bbaa16907cb22d82'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.094811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] The token cache is being invalidate because project f5901df4ad1643d99b5884468b91f94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:09.095106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.095758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.097314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 900/1803] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:08 2022] DELETE /identity/v3/projects/f5901df4ad1643d99b5884468b91f94e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:09.101107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:09.102395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:09.109622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 904/1804] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:08 2022] DELETE /identity/v3/users/7b9c5c3ef19d4bc6bbaa16907cb22d82 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:09.182536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d6311-4e7c-408b-8f23-a4d0d55d2b34 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:09.245057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:09.245889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.246178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.246456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] PATH_INFO: `/v3/users/0de213483ca54b54810ed4f20a6675a7` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.247667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de213483ca54b54810ed4f20a6675a7)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:09.249644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:09.335758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de213483ca54b54810ed4f20a6675a7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.347919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de213483ca54b54810ed4f20a6675a7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.348847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de213483ca54b54810ed4f20a6675a7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.349905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de213483ca54b54810ed4f20a6675a7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.354195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:09.374446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c700608c-e000-4bb1-92a4-11a86af84dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de213483ca54b54810ed4f20a6675a7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.393150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 901/1805] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:09 2022] DELETE /identity/v3/users/0de213483ca54b54810ed4f20a6675a7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:09.467048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:09.468629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:09.485602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c083a-1ec6-4045-b274-442e5ede2c7a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:09.594290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:09.595076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.595362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.596041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] PATH_INFO: `/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.597715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1287b0be3ab443c48ee03a35dcafc3c8)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:09.615723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:09.709423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1287b0be3ab443c48ee03a35dcafc3c8', 'request_id': 'req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.723603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1287b0be3ab443c48ee03a35dcafc3c8', 'request_id': 'req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.728525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:09.736624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1287b0be3ab443c48ee03a35dcafc3c8', 'request_id': 'req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.757727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] The token cache is being invalidate because project 1287b0be3ab443c48ee03a35dcafc3c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:09.758392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.759427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-66d5f3c0-1efd-442b-a3a2-3de8847ce89c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:09.762186 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 905/1806] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:09 2022] DELETE /identity/v3/projects/1287b0be3ab443c48ee03a35dcafc3c8 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:09.769344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.769344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.769344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.797551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:09.799593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:09.958477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:09.958975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:09.959319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:09.967267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 902/1807] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:09.988895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:09.990291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.001094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 906/1808] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:10.007594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.022861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8f7da-e7ea-490b-b65f-f05898649d02 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.029265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.029973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.030258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.030531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.031812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:10.031812 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.033615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.056793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.064718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 903/1809] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:10.077571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140670f0-e189-4e52-82d5-01a34de6817a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.085113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.086084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.086377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.086653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.087335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.090180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.124396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 907/1810] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/projects => generated 381 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:10.128497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.140440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cb8f6-9b3f-45fd-84c6-4b425b390075 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.255871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.256606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.256888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.257167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.257913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.261097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.304679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 904/1811] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:10.309879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.319997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.325136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.328045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad204-e6ce-48d1-ad91-841540ebe157 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.342138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.342524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.342918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.343303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.345037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.346273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.354664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1e916-189b-4a14-a2cf-763323db2147 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.361118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.361912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.362392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.362844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] PATH_INFO: `/v3/projects/19447f679aa942bd83e986e0649c1403` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.364245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19447f679aa942bd83e986e0649c1403)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.366422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.392099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 908/1812] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:10.393912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.420080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1249d-22fc-438f-9ee6-7b2360a7f4f5 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.436054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:10.442790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19447f679aa942bd83e986e0649c1403', 'request_id': 'req-b463025c-93d4-49e1-ae8f-18e720ad4626'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:10.455917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19447f679aa942bd83e986e0649c1403', 'request_id': 'req-b463025c-93d4-49e1-ae8f-18e720ad4626'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:10.469741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19447f679aa942bd83e986e0649c1403', 'request_id': 'req-b463025c-93d4-49e1-ae8f-18e720ad4626'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:10.486776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] The token cache is being invalidate because project 19447f679aa942bd83e986e0649c1403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:10.487243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:10.488034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b463025c-93d4-49e1-ae8f-18e720ad4626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:10.495029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 905/1813] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:10 2022] DELETE /identity/v3/projects/19447f679aa942bd83e986e0649c1403 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:10.499052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd825c6-dd2b-47a5-95b6-7b256a5f4917 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.499649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd825c6-dd2b-47a5-95b6-7b256a5f4917 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.501040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd825c6-dd2b-47a5-95b6-7b256a5f4917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.546460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cdd825c6-dd2b-47a5-95b6-7b256a5f4917 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:10.548094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cdd825c6-dd2b-47a5-95b6-7b256a5f4917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.567016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.567796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.568231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.568630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.569422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.571116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.583547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 909/1814] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:10.587428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.594356 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.612604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace78a3-9893-40d1-97ec-8bfdcf5a4e9c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.637470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.638541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.639012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.639450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] PATH_INFO: `/v3/projects/ad226164a496400cb0255b9bc3dd1e2c/users/cb8ae716cd534bcfac38ffee5e055a49/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.667968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 906/1815] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:10.669742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.670144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.670518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.690674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad226164a496400cb0255b9bc3dd1e2c, user_id=cb8ae716cd534bcfac38ffee5e055a49, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.692710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1e1b4-dc11-4a27-9df9-8aafc704acec admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.698282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:10.699805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.716911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 907/1816] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:10.719278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.721640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 910/1817] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:10 2022] PUT /identity/v3/projects/ad226164a496400cb0255b9bc3dd1e2c/users/cb8ae716cd534bcfac38ffee5e055a49/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:10.733811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.734296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.734763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.743334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8335442-9763-4208-b5b6-3d7ed4b110c0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.774223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] MFA Rules not processed for user `cb8ae716cd534bcfac38ffee5e055a49`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:10.776255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:10.852625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.853281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.854241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.855445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.856630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:10.857478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.860474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.882085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 908/1818] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:10.883456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 911/1819] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:10.890433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.891251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.907550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7cf7-bd04-4759-9b0a-f47c71d32cb2 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.909146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43145fd-dc48-4471-a871-2239b8cdaa24 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:10.914573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.914753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:10.915223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.915505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.915783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.917108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.917450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:10.918003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:10.918479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:10.919459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:10.920642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.922907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:10.927595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:10.968080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 912/1820] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/projects => generated 383 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:10.970212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:10.972199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 909/1821] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/projects => generated 387 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:10.976169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.041138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:11.068001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb2ea-9dae-49ef-b5f8-15dd1dc92a97 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.074777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.075477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.075824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.076081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.076796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.078936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.079076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86a917-90b5-4ab4-a261-67f6c8bab947 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.085240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.086542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.086934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.087353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.089251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.094331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.124697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 913/1822] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:11.131031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.167434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea62fb-b0c8-44b7-9a9e-99fbc0a5c97f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.177156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 910/1823] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:10 2022] POST /identity/v3/users => generated 294 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:11.182102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.189330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.190337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.190796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.191352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.192573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.198957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.205314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d53e2a-5d50-4c7b-9b48-b010be351462 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.223670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 914/1824] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:11.223967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.223967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.224634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.229015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.231932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.232790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.237768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.255037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15e51a-1add-4ce2-a189-787aeead980c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.269625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.271052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.271346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.271629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] PATH_INFO: `/v3/projects/39870be5a5cc45b4924173b01e1588fe/users/9135c03dd7ab4c62a255b5042d7b5290/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.274762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 911/1825] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:11.280630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.313535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9acbc0-120d-421a-80cb-aa91475b2c4d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.318193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39870be5a5cc45b4924173b01e1588fe, user_id=9135c03dd7ab4c62a255b5042d7b5290, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.324561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.332573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.333396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.334316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.335069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.336036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.338395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.356703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 915/1826] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:11 2022] PUT /identity/v3/projects/39870be5a5cc45b4924173b01e1588fe/users/9135c03dd7ab4c62a255b5042d7b5290/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:11.360876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.371645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 912/1827] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:11.376902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.404715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa11db0-ac2c-4403-a3ee-d1a4cf734b88 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.581244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941fd79-7552-4b48-bd99-b8762b7a9ea6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.592471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.595281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.597396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.597875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.601013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.603184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.607281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.608087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.608528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.608906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] PATH_INFO: `/v3/projects/ccedce042b1249f2ac4e14e42ca7e3f5/users/00af20ff149f474e90c922e379c7acc7/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.622333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 916/1828] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:11 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:11.628991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.634084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccedce042b1249f2ac4e14e42ca7e3f5, user_id=00af20ff149f474e90c922e379c7acc7, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.637756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebe31a-e218-46bf-b73b-15284291b1d4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.649616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c3d93b-2eda-40a5-869d-a46ee2e68d75 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:11.661236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:11.663392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.663769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.664272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] PATH_INFO: `/v3/projects/39870be5a5cc45b4924173b01e1588fe/users/9135c03dd7ab4c62a255b5042d7b5290/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.668073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39870be5a5cc45b4924173b01e1588fe, user_id=9135c03dd7ab4c62a255b5042d7b5290, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:11.668307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 913/1829] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:11 2022] PUT /identity/v3/projects/ccedce042b1249f2ac4e14e42ca7e3f5/users/00af20ff149f474e90c922e379c7acc7/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:11.679417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.679712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.679993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.691607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfe3a6-3915-4328-8794-7841d5a2cd21 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:11.723498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 917/1830] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:11 2022] PUT /identity/v3/projects/39870be5a5cc45b4924173b01e1588fe/users/9135c03dd7ab4c62a255b5042d7b5290/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:11.748394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:11.748394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:11.748394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:11.753309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] MFA Rules not processed for user `00af20ff149f474e90c922e379c7acc7`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:11.754497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:11.809005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] MFA Rules not processed for user `9135c03dd7ab4c62a255b5042d7b5290`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:11.810266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:11.837794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:11.904284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 914/1831] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:11.910015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.959956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:11.980494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 918/1832] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:11.995125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:11.997069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:12.043146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e2c80-25dc-455b-b532-d2dbfd1eccb6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.052146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.053168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.053572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.053852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.054783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.058041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.073393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320756ad-2fb7-4ca6-bf21-72470839b288 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.078511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.079363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.079792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.080210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.091710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.094010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.123461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 919/1833] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:12.128919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 915/1834] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:11 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:12.144071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.156774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e0669-a42d-46ec-896b-7d5e723c94a0 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.266457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.285314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.286143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.287123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.288589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.289809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.292015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.320869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 920/1835] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:12 2022] POST /identity/v3/users => generated 300 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:12.325337 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.336555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276f1ea-9f5f-49a5-821d-67ef811ac2bc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.343484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.345176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.345574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.345996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.346943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.349094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.349522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e35a6-f132-48eb-bdd8-5269fd901492 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.354165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.354890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.355176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.355447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.362978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 921/1836] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:12.367304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.396004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd06a86-78f0-43c0-ad63-41d76a8881dc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.411291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.412562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.413256 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.414648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] PATH_INFO: `/v3/projects/7e6ab82a3eea4499a208baa4765181b1/users/97644d8b15754a54884f424f276fcc81/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.455523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ab82a3eea4499a208baa4765181b1, user_id=97644d8b15754a54884f424f276fcc81, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.458399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.477286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 922/1837] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:12 2022] PUT /identity/v3/projects/7e6ab82a3eea4499a208baa4765181b1/users/97644d8b15754a54884f424f276fcc81/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:12.482045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.482509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.484077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.498242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 916/1838] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:12.503028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:12.601418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c51c1-544e-4116-86d9-53960a86aae9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.618059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.618895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.620975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.621357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.622403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.624295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d66b59-b558-4621-b3fa-fdab7993e29a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.639322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 923/1839] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:12.643348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.661207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e437b2c-e615-4aaf-a9b0-95e88aa92118 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:12.678479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:12.679917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.680206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.680513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] PATH_INFO: `/v3/projects/7e6ab82a3eea4499a208baa4765181b1/users/97644d8b15754a54884f424f276fcc81/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.689291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ab82a3eea4499a208baa4765181b1, user_id=97644d8b15754a54884f424f276fcc81, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:12.689291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:12.708726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 917/1840] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:12 2022] PUT /identity/v3/projects/7e6ab82a3eea4499a208baa4765181b1/users/97644d8b15754a54884f424f276fcc81/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:12.717280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:12.717561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:12.717840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:12.843445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] MFA Rules not processed for user `97644d8b15754a54884f424f276fcc81`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:12.845635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:12.918036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.949134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 924/1841] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:12.962210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:12.964230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.020560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:13.052113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3cdff-f440-4e91-bd18-533f380708b9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.058818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.059544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.059824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.060093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] PATH_INFO: `/v3/users/cb8ae716cd534bcfac38ffee5e055a49` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.061386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb8ae716cd534bcfac38ffee5e055a49)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.063542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.079293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4938d-a800-4323-81a5-3910e83b04df None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.085381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.086060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.086343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.086625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.101769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.108386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6297ecf-b065-4017-b5c8-19716e160759 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.119643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 925/1842] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:13.139701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb8ae716cd534bcfac38ffee5e055a49'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.174730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb8ae716cd534bcfac38ffee5e055a49'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.175798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb8ae716cd534bcfac38ffee5e055a49'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.177406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb8ae716cd534bcfac38ffee5e055a49'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.200484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb8ae716cd534bcfac38ffee5e055a49'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.229510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 918/1843] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:12 2022] DELETE /identity/v3/users/cb8ae716cd534bcfac38ffee5e055a49 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:13.235707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.236143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.236420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.284048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:13.285924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.328666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:13.329069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:13.332547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.384155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 926/1844] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:13.395201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:13.397085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.428626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11e30-c615-431a-9a2e-c48d4c3d84c2 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.434438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.436847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.437206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.437558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] PATH_INFO: `/v3/users/00af20ff149f474e90c922e379c7acc7` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.440325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00af20ff149f474e90c922e379c7acc7)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.443101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.486047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de612d-f715-4445-b9c9-a8f814367f26 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.491214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.492160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.492589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.492931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.504783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.506478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99668fad-092a-45fe-88fd-d0af2121cab8 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.513707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00af20ff149f474e90c922e379c7acc7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.522220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 927/1845] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:13.526622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00af20ff149f474e90c922e379c7acc7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.527384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00af20ff149f474e90c922e379c7acc7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.528109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00af20ff149f474e90c922e379c7acc7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.548071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:13.558063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00af20ff149f474e90c922e379c7acc7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:13.582540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 919/1846] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:13 2022] DELETE /identity/v3/users/00af20ff149f474e90c922e379c7acc7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:13.588141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.588699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.589654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.615233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:13.616506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.707735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 928/1847] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:13.723205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:13.723205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.821164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db662d7e-a20d-44ff-be6b-c5fed7ec1da1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.828938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.829593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.829868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.830246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.841809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.843656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.854594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 920/1848] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:13.894854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:13.896311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:13.914686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91d809-d5cc-41a7-ac18-bd4dc116f9cc None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:13.922371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:13.923162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:13.923520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:13.923873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] PATH_INFO: `/v3/projects/ad226164a496400cb0255b9bc3dd1e2c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:13.925660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad226164a496400cb0255b9bc3dd1e2c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:13.927594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:13.996843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad226164a496400cb0255b9bc3dd1e2c', 'request_id': 'req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.007392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad226164a496400cb0255b9bc3dd1e2c', 'request_id': 'req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.016628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad226164a496400cb0255b9bc3dd1e2c', 'request_id': 'req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.025670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:14.035320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] The token cache is being invalidate because project ad226164a496400cb0255b9bc3dd1e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:14.035644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.036789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.039203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 929/1849] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:13 2022] DELETE /identity/v3/projects/ad226164a496400cb0255b9bc3dd1e2c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:14.044683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:14.061097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7dc1a-556b-4070-8f1a-ea95c174d280 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:14.135122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:14.136094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:14.136420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:14.136705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] PATH_INFO: `/v3/users/9135c03dd7ab4c62a255b5042d7b5290` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:14.137998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9135c03dd7ab4c62a255b5042d7b5290)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:14.139956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:14.199506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:14.203351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:14.236523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f0d41-f46c-489d-90ef-a91bf55e42cb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:14.253221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9135c03dd7ab4c62a255b5042d7b5290'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.263240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:14.266136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:14.266310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:14.269710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] PATH_INFO: `/v3/projects/ccedce042b1249f2ac4e14e42ca7e3f5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:14.275359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccedce042b1249f2ac4e14e42ca7e3f5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:14.277553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:14.279663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9135c03dd7ab4c62a255b5042d7b5290'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.281720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9135c03dd7ab4c62a255b5042d7b5290'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.282714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9135c03dd7ab4c62a255b5042d7b5290'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.328366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9135c03dd7ab4c62a255b5042d7b5290'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.366101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 921/1850] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:14 2022] DELETE /identity/v3/users/9135c03dd7ab4c62a255b5042d7b5290 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:14.388787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccedce042b1249f2ac4e14e42ca7e3f5', 'request_id': 'req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.402634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccedce042b1249f2ac4e14e42ca7e3f5', 'request_id': 'req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.414973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccedce042b1249f2ac4e14e42ca7e3f5', 'request_id': 'req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.432062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] The token cache is being invalidate because project ccedce042b1249f2ac4e14e42ca7e3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:14.432582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.434034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.435322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 930/1851] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:14 2022] DELETE /identity/v3/projects/ccedce042b1249f2ac4e14e42ca7e3f5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:14.440612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:14.442694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:14.455980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:14.633207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5191790-94fb-4484-aef4-b3b97db0e9e7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:14.733709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:14.734872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:14.735589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:14.736309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] PATH_INFO: `/v3/users/97644d8b15754a54884f424f276fcc81` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:14.740332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=97644d8b15754a54884f424f276fcc81)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:14.742904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:14.802869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97644d8b15754a54884f424f276fcc81'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.813782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97644d8b15754a54884f424f276fcc81'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.814455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97644d8b15754a54884f424f276fcc81'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.815148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97644d8b15754a54884f424f276fcc81'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.819215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:14.820693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:14.832430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d2f13c39-6aed-4fbc-9cb6-009d0518e2fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97644d8b15754a54884f424f276fcc81'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.841902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a0d9c-e1fe-47ff-9f41-8769b719fc4c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:14.852892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:14.854380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:14.854825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:14.855248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] PATH_INFO: `/v3/projects/39870be5a5cc45b4924173b01e1588fe` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:14.859597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 922/1852] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:14 2022] DELETE /identity/v3/users/97644d8b15754a54884f424f276fcc81 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:14.862927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39870be5a5cc45b4924173b01e1588fe)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:14.866951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:14.925822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39870be5a5cc45b4924173b01e1588fe', 'request_id': 'req-8d3b1f32-43bd-4f8a-b543-7ff46002b732'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.935864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39870be5a5cc45b4924173b01e1588fe', 'request_id': 'req-8d3b1f32-43bd-4f8a-b543-7ff46002b732'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.944271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39870be5a5cc45b4924173b01e1588fe', 'request_id': 'req-8d3b1f32-43bd-4f8a-b543-7ff46002b732'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.956619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] The token cache is being invalidate because project 39870be5a5cc45b4924173b01e1588fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:14.956935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.957638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8d3b1f32-43bd-4f8a-b543-7ff46002b732 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:14.959378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 931/1853] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:14 2022] DELETE /identity/v3/projects/39870be5a5cc45b4924173b01e1588fe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:14.964868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:14.965163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:14.965460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:14.985801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:14.987160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:15.014702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:15.051965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 923/1854] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:15.059947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.060248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.060541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.081393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:15.082601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:15.091609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 932/1855] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:15.095835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.105619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ddc15-f3c8-422f-a6c3-06521f10e8ae None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.110224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.110781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.111076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.111334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.114707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:15.114707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.114707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.126900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 924/1856] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:15.131577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.142199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75629b19-3323-412d-85db-a50ebff9c48a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.147100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.147723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.148008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.148284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.148989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.151206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.184645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 933/1857] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:15.186461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.198208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb4841-d73a-49b7-971c-5e96ef4af067 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.280033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.281220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.281507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.281790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.282443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.284400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.311059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 925/1858] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:15.315816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.327790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1b170-569c-490d-aff3-ae6d7a12bc46 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.333426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.334063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.334341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.334611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.335299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.337065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.351442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 934/1859] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:15.352769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.363739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657bfd3-b7b8-4f69-8be1-3a0d4e5f16ec admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.370358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.371008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.371277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.371591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.372246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.374044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2293e-84d6-4858-be11-35dc89f8ddf8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.392441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 926/1860] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:15.394591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.407482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6ab9-064f-4086-88a5-432ac762ba80 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.412941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.413636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.414592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.414980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] PATH_INFO: `/v3/projects/a423003a15b54b36abdede9fc0dd5c49/users/36769065ad6d4effbb778b69ab92ecd3/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.434797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a423003a15b54b36abdede9fc0dd5c49, user_id=36769065ad6d4effbb778b69ab92ecd3, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.438452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.451837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:15.468448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 935/1861] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:15 2022] PUT /identity/v3/projects/a423003a15b54b36abdede9fc0dd5c49/users/36769065ad6d4effbb778b69ab92ecd3/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:15.473279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.473584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.473843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.534321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] MFA Rules not processed for user `36769065ad6d4effbb778b69ab92ecd3`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:15.535381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:15.633108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 927/1862] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:15.635221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.743427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad68fd-1f51-4c7d-b0bd-46d184112ee0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.749440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.750083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.750363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.750631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.752031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.754357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.801626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 936/1863] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:15.805901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.829611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab68507-8c41-4551-961b-50344cef3e65 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.864074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:15.915118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.922005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.922005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.922005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.922005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.922005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.956510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 928/1864] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:15 2022] POST /identity/v3/users => generated 299 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:15.957810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:15.970021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af8704-d32a-4378-80b8-95ea612e050b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:15.975067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:15.975703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:15.975989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:15.976266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:15.976999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:15.979174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:15.996810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 937/1865] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:15.998545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.012130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cc2ff-8133-4ed8-a2ba-483924c3ae4d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.020820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.021688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.022024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.022798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] PATH_INFO: `/v3/projects/1182be89159c45588a2baefa10099149/users/b4fad1a90f7e4690a1226b335a95710b/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.046692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.050707 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.065787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1182be89159c45588a2baefa10099149, user_id=b4fad1a90f7e4690a1226b335a95710b, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.077721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.088527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de027cbd-29af-48a9-b23d-ddf56b96ea66 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.100119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.102950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.103570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.104220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] PATH_INFO: `/v3/projects/7e6ab82a3eea4499a208baa4765181b1` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.105960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6ab82a3eea4499a208baa4765181b1)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.108412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 929/1866] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:15 2022] PUT /identity/v3/projects/1182be89159c45588a2baefa10099149/users/b4fad1a90f7e4690a1226b335a95710b/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:16.110095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.112234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.206924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed9d75-55d6-455f-a80c-14cbceaae6cd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.217147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6ab82a3eea4499a208baa4765181b1', 'request_id': 'req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:16.232803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6ab82a3eea4499a208baa4765181b1', 'request_id': 'req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:16.246463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6ab82a3eea4499a208baa4765181b1', 'request_id': 'req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:16.270740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] The token cache is being invalidate because project 7e6ab82a3eea4499a208baa4765181b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:16.271044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:16.278227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a7867a39-70a3-4764-bbc9-f9ab6ca4f2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:16.288366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 938/1867] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:16 2022] DELETE /identity/v3/projects/7e6ab82a3eea4499a208baa4765181b1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:16.298117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557566c-b066-493b-ba6d-94fae563951b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.300011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557566c-b066-493b-ba6d-94fae563951b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.303363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557566c-b066-493b-ba6d-94fae563951b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.351120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.352097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.352496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.352633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e557566c-b066-493b-ba6d-94fae563951b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:16.353132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.354488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.355574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e557566c-b066-493b-ba6d-94fae563951b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.357164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.372478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 930/1868] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:16.375489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.382084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.385452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 939/1869] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:16.389691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.390061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.390267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.402890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef7a87-072f-4a80-bc95-475ce3822ffe admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.412307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.413048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.413333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.413614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] PATH_INFO: `/v3/projects/1182be89159c45588a2baefa10099149/users/b4fad1a90f7e4690a1226b335a95710b/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.414738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:16.415629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1182be89159c45588a2baefa10099149, user_id=b4fad1a90f7e4690a1226b335a95710b, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.416782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.419419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f552240-7811-460b-9b24-c9fcc314084c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.426875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 940/1870] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:16.433689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 931/1871] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:16 2022] PUT /identity/v3/projects/1182be89159c45588a2baefa10099149/users/b4fad1a90f7e4690a1226b335a95710b/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:16.446134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.446558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.446711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.447055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.458527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad6c6f-e9ab-4944-aead-fab4a9f4ef55 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.544188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] MFA Rules not processed for user `b4fad1a90f7e4690a1226b335a95710b`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:16.546495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.559899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:16.560620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:16.586431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.586431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.586732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.587093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.588192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:16.588642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.590697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.609858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 941/1872] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:16.615329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.627695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed2468-6209-4488-90f9-e209ae47b4ae admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.633507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.635820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.636305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.636904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.638606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.643183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.685761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 932/1873] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:16.687064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.689388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.698442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 942/1874] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:16 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:16.701283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.767277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2f690-71d2-4127-98cf-e750ffb20375 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.785748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b3eae7-a957-4862-bf7f-bb8865c56383 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.792748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.793435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.793719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.793995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.794659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.796890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.844510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 943/1875] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:16 2022] POST /identity/v3/users => generated 286 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:16.844699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.844699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.844699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.844699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.845041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:16.869645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.894661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8acaf-6a4c-47a5-8170-cbb4d0fe364b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.907583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.908221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.908677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.908967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.909651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.911271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.923713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 944/1876] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:16.928415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.939264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba89c9-b22c-444c-ab64-ab01b30a1514 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.939432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.941646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.944136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.944967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.945255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.945604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:16.946301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:16.948100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c4e12-4778-4d80-925b-02983886d91d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:16.955353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 933/1877] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:16.960184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 945/1878] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:16.965383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:16.979833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262cc0-0fe5-4a60-b7a3-b46f11b8be60 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:16.984985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:16.985672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:16.985952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:16.986293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] PATH_INFO: `/v3/projects/96efe0a69fc44794b2c5358528fa63b2/users/9906d7d342664ae6b1e4a3f8df5b5662/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:17.007071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=96efe0a69fc44794b2c5358528fa63b2, user_id=9906d7d342664ae6b1e4a3f8df5b5662, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:17.010810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:17.013404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:17.029517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 934/1879] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:16 2022] PUT /identity/v3/projects/96efe0a69fc44794b2c5358528fa63b2/users/9906d7d342664ae6b1e4a3f8df5b5662/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:17.042831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2b8a-375b-4c50-973a-de414ab369c3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:17.043136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2b8a-375b-4c50-973a-de414ab369c3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:17.043392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c2b8a-375b-4c50-973a-de414ab369c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:17.079145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-304c2b8a-375b-4c50-973a-de414ab369c3 None None] MFA Rules not processed for user `9906d7d342664ae6b1e4a3f8df5b5662`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:17.080527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-304c2b8a-375b-4c50-973a-de414ab369c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:17.168053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 946/1880] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:17.184146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:17.186871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:17.307138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172b566-9b99-447e-bba5-309bb5173b3b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:17.312148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:17.312804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:17.313093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:17.313345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:17.323950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:17.326145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee758eea-bcee-41ae-a578-76c6d02985db service glance] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:17.338394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 935/1881] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:18.203076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:18.203625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:18.203726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:18.205174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 947/1882] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:55:18.220526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:18.221086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:18.221261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:18.242920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:18.244851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:18.328816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 936/1883] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:55:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:18.340856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:18.342510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:18.359634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c766e-ae52-4c18-85e5-6f833e860b46 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:18.365101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:18.365708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:18.365999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:18.366276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:18.375978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:18.377587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d8dc2-b832-4e6a-8260-68eb18a6b87a service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:18.383603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:18.390125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 948/1884] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:19.519878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:19.521441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:19.628593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2e320-2437-4cef-a52e-9b2a330e13a1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:19.634299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:19.635167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:19.636369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:19.636369 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] PATH_INFO: `/v3/users/080299acab69406f918ef3fb3a5edab9` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:19.637179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=080299acab69406f918ef3fb3a5edab9)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:19.639408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:19.652473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:19.700051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080299acab69406f918ef3fb3a5edab9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:19.715138 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080299acab69406f918ef3fb3a5edab9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:19.715790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080299acab69406f918ef3fb3a5edab9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:19.716427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080299acab69406f918ef3fb3a5edab9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:19.732702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080299acab69406f918ef3fb3a5edab9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:19.746559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 937/1885] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:19 2022] DELETE /identity/v3/users/080299acab69406f918ef3fb3a5edab9 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:19.750962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:19.751474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:19.751788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:19.778720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:19.780148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:19.858027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 949/1886] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:19.893132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:19.895352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:19.960206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be5257-0faf-4a11-9d48-dd0c4fa96c6b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:19.968184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:19.969047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:19.969393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:19.969747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:19.988326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:19.991388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:20.011177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 938/1887] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:20.374743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:20.376388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:20.403603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ff472-d524-4063-ae8b-c9aac97b4160 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:20.415307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:20.416416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:20.416731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:20.416953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] PATH_INFO: `/v3/projects/1f31314d8f5b4fdcaa04a5115000baef` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:20.418384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f31314d8f5b4fdcaa04a5115000baef)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:20.420332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:20.488541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f31314d8f5b4fdcaa04a5115000baef', 'request_id': 'req-0054ec46-72a5-433f-a066-53aad46d0b3d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.497947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f31314d8f5b4fdcaa04a5115000baef', 'request_id': 'req-0054ec46-72a5-433f-a066-53aad46d0b3d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.505965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f31314d8f5b4fdcaa04a5115000baef', 'request_id': 'req-0054ec46-72a5-433f-a066-53aad46d0b3d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.520055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] The token cache is being invalidate because project 1f31314d8f5b4fdcaa04a5115000baef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:20.520547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.521540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.523630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 950/1888] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:20 2022] DELETE /identity/v3/projects/1f31314d8f5b4fdcaa04a5115000baef => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:20.530766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:20.547462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0101f-80f1-476e-84e6-abac0f0bb3db service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:20.640875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:20.641563 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:20.641995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:20.642383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] PATH_INFO: `/v3/users/e3adf62464ca472295c99b059a77eb4f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:20.643909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3adf62464ca472295c99b059a77eb4f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:20.646099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:20.688131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:20.703577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3adf62464ca472295c99b059a77eb4f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.712232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3adf62464ca472295c99b059a77eb4f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.713559 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3adf62464ca472295c99b059a77eb4f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.714200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3adf62464ca472295c99b059a77eb4f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.727699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e5013775-0dfe-43d5-8d20-acdcd0433d7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3adf62464ca472295c99b059a77eb4f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:20.740945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 939/1889] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:20 2022] DELETE /identity/v3/users/e3adf62464ca472295c99b059a77eb4f => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:21.226025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.233538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:21.257004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0054ec46-72a5-433f-a066-53aad46d0b3d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.338101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.338836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.339121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.339399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] PATH_INFO: `/v3/projects/846fa8e91e804b6d935488b6bfb102c6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.340627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=846fa8e91e804b6d935488b6bfb102c6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.342626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.380293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:21.402691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846fa8e91e804b6d935488b6bfb102c6', 'request_id': 'req-9ea8119e-866c-4c89-9d4e-715ecfb04aea'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:21.415229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846fa8e91e804b6d935488b6bfb102c6', 'request_id': 'req-9ea8119e-866c-4c89-9d4e-715ecfb04aea'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:21.421301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846fa8e91e804b6d935488b6bfb102c6', 'request_id': 'req-9ea8119e-866c-4c89-9d4e-715ecfb04aea'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:21.434118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] The token cache is being invalidate because project 846fa8e91e804b6d935488b6bfb102c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:21.434410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:21.435118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9ea8119e-866c-4c89-9d4e-715ecfb04aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:21.436748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 951/1890] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:21 2022] DELETE /identity/v3/projects/846fa8e91e804b6d935488b6bfb102c6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:21.443103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.443948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.444367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.466597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:21.467857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:21.541353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 940/1891] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:21.551258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.551732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.552198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.579895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:21.581190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:21.589414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 952/1892] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:21.594065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.605592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6f0c0-c150-4036-8c87-81d9f4046319 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.610879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.612016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.612318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.612985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.613674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:21.614181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.617017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.632242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 941/1893] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:55:21.636890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.646401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b10e97-bc75-4774-a2e3-5a6e63248162 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.651025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.651672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.651922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.652205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.652920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.655466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.674715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.676840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:21.690798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 953/1894] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:21 2022] POST /identity/v3/projects => generated 399 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:21.694986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.745413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05334b4-b88c-46ed-b032-8426dd83bf36 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.786962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9da4e09-0ade-425c-9ac2-38a2594d9193 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.794173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.794797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.795078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.795350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.796016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.797981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.808694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.809350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.809625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.809897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.811717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:21.819635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:21.822360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 954/1895] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:21 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:21.827169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.837534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d8059-0c67-4c8e-bf2d-bf8b685bc3ab admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.842988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.843901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.844560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.844853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.845552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.848454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.862560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 955/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:21.866980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.880404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2784-ef88-4386-96d5-8a1ffb7b62d0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.886221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.887044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.887175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.887400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.888148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.890990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.902110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.903551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c7352-5ff1-4701-aa76-6a08918bf532 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.904688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 956/1897] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:21.908859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:21.916392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 942/1898] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:21.920157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75602a31-9837-4274-bcbb-6abab5aec4c1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:21.926495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:21.927551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.927980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.928603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] PATH_INFO: `/v3/projects/1435830c5d2c41df84af51afb57db467/users/fc53de9ebc9a46d4964deb2cbdd8b385/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:21.946465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1435830c5d2c41df84af51afb57db467, user_id=fc53de9ebc9a46d4964deb2cbdd8b385, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:21.949538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:21.983691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 957/1899] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:21 2022] PUT /identity/v3/projects/1435830c5d2c41df84af51afb57db467/users/fc53de9ebc9a46d4964deb2cbdd8b385/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:21.995598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:21.996316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:21.996839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.029853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] MFA Rules not processed for user `fc53de9ebc9a46d4964deb2cbdd8b385`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:22.031618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:22.134488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 943/1900] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:22.139364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.219768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=6,keystone:SELECT=88,keystone:INSERT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:22.278780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89506385-93b6-42de-b9a0-e4782d684a2e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.284722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.285411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.285701 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.286244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.288613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.289931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.368824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 958/1901] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:22 2022] POST /identity/v3/projects => generated 401 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:22.373128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.391292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d97226-dc1c-47c7-9953-88bfeaf78ba3 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.523359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.526128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.528058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.528374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.529165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.533740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.576800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 944/1902] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:22 2022] POST /identity/v3/users => generated 307 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:22.580718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.613047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81772c3a-2ccd-4240-9ada-2f287bd4568e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.620108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.620841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.621135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.621461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.622170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.623751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.638638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 959/1903] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:22.643082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.655150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3fa8f-a730-4b57-ba2f-9d01a3f96965 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.660067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.660956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.661272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.661523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] PATH_INFO: `/v3/projects/71852375d53a4482ab9e0e81ca2f2abb/users/4a506d4910a0452e944f56515d1ea23d/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.682377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=71852375d53a4482ab9e0e81ca2f2abb, user_id=4a506d4910a0452e944f56515d1ea23d, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.685117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.694972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:22.705373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 945/1904] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:22 2022] PUT /identity/v3/projects/71852375d53a4482ab9e0e81ca2f2abb/users/4a506d4910a0452e944f56515d1ea23d/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:22.706543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.719022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81765a26-4df9-4d1c-921f-ae8f0074d724 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.778837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.779468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.779773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.780050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.780832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.782501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3834736-5a87-4764-984f-7da9d5650d44 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.795874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 960/1905] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:22.799551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:22.810845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd312fa4-05bb-48dc-a725-d3ee7ccbbc7c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:22.815696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:22.816398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.816681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.816953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] PATH_INFO: `/v3/projects/71852375d53a4482ab9e0e81ca2f2abb/users/4a506d4910a0452e944f56515d1ea23d/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.818948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71852375d53a4482ab9e0e81ca2f2abb, user_id=4a506d4910a0452e944f56515d1ea23d, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:22.821822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:22.832413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 946/1906] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:22 2022] PUT /identity/v3/projects/71852375d53a4482ab9e0e81ca2f2abb/users/4a506d4910a0452e944f56515d1ea23d/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:22.844425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:22.844720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:22.844996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:22.873744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] MFA Rules not processed for user `4a506d4910a0452e944f56515d1ea23d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:22.874952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:22.943712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 961/1907] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:22 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:22.955048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:23.014817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027cee2a-9bf5-4da0-ad97-f749760759b0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:23.018344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:23.018953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:23.019234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:23.019505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:23.029098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:23.030555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:23.040254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 947/1908] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:22 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:23.130291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:23.139828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468000a-a689-42d1-925b-e2876683f0ee None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:23.143452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:23.144054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:23.144331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:23.144631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:23.178222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:23.214746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:23.216179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:23.226526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 962/1909] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:23 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:27.476205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:27.476205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:27.564785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006aa27d-e344-4634-a471-de934e5ce6ec service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:27.570538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:27.571443 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:27.573231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:27.573231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] PATH_INFO: `/v3/users/36769065ad6d4effbb778b69ab92ecd3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:27.574934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36769065ad6d4effbb778b69ab92ecd3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:27.577958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:27.711729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36769065ad6d4effbb778b69ab92ecd3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.727572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36769065ad6d4effbb778b69ab92ecd3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.728479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36769065ad6d4effbb778b69ab92ecd3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.729105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36769065ad6d4effbb778b69ab92ecd3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.754115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4c3312d2-57a0-4035-abc2-9f8bfa9cc708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36769065ad6d4effbb778b69ab92ecd3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.760193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:27.776059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fe9dbc-56d4-4f95-9bb8-d1bcce409ad6 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:27.782407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:27.783176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:27.783529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:27.783881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] PATH_INFO: `/v3/users/fc53de9ebc9a46d4964deb2cbdd8b385` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:27.785247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc53de9ebc9a46d4964deb2cbdd8b385)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:27.787903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:27.791381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 948/1910] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:27 2022] DELETE /identity/v3/users/36769065ad6d4effbb778b69ab92ecd3 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:27.795960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:27.796269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:27.796582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:27.811411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:27.819100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:27.820673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:27.879544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc53de9ebc9a46d4964deb2cbdd8b385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.881418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc53de9ebc9a46d4964deb2cbdd8b385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.896991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc53de9ebc9a46d4964deb2cbdd8b385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.916506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc53de9ebc9a46d4964deb2cbdd8b385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.918390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-86372b3c-fd6e-4136-b170-5253d8ea2d5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc53de9ebc9a46d4964deb2cbdd8b385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:27.920829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 949/1911] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:27.930642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:27.932426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:27.961366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 963/1912] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:27 2022] DELETE /identity/v3/users/fc53de9ebc9a46d4964deb2cbdd8b385 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:27.965374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:27.966102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:27.966762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:28.009300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:28.011185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:28.019242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a587607-92bb-4764-8add-c3c8b2da9237 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:28.103059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:28.103850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:28.104124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:28.104481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:28.115157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 964/1913] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:28.126280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:28.138155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c31294-4632-4563-8952-f2b30ce02654 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:28.141411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:28.142677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:28.142970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:28.143246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:28.155289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:28.157769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:28.169955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 965/1914] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:28.218550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:28.221377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:28.247891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 950/1915] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:29.009910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:29.015621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:29.049196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c907f86-16b5-4844-aa31-005afa5c0f23 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:29.084706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:29.085383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:29.085674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:29.085967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] PATH_INFO: `/v3/projects/1435830c5d2c41df84af51afb57db467` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:29.096426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1435830c5d2c41df84af51afb57db467)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:29.103473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:29.157893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:29.244704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1435830c5d2c41df84af51afb57db467', 'request_id': 'req-cd2aa48c-639c-483d-aaae-2159c931586e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.261276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1435830c5d2c41df84af51afb57db467', 'request_id': 'req-cd2aa48c-639c-483d-aaae-2159c931586e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.278003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1435830c5d2c41df84af51afb57db467', 'request_id': 'req-cd2aa48c-639c-483d-aaae-2159c931586e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.284858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:29.288152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:29.306849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] The token cache is being invalidate because project 1435830c5d2c41df84af51afb57db467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:29.307118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.313019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680797e-ce05-45fd-917d-a437c71bcf01 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:29.316836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.318607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 966/1916] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:29 2022] DELETE /identity/v3/projects/1435830c5d2c41df84af51afb57db467 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:29.323275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:29.324716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:29.330267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:29.425959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:29.426790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:29.427141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:29.427492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] PATH_INFO: `/v3/projects/a423003a15b54b36abdede9fc0dd5c49` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:29.430766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a423003a15b54b36abdede9fc0dd5c49)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:29.434359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:29.473937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2aa48c-639c-483d-aaae-2159c931586e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:29.497973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:29.500929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:29.502488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:29.503224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] PATH_INFO: `/v3/users/4a506d4910a0452e944f56515d1ea23d` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:29.506876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a423003a15b54b36abdede9fc0dd5c49', 'request_id': 'req-2efffc30-71ff-4460-ad5e-b165a44954cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.507314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a506d4910a0452e944f56515d1ea23d)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:29.514288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:29.519314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a423003a15b54b36abdede9fc0dd5c49', 'request_id': 'req-2efffc30-71ff-4460-ad5e-b165a44954cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.537427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a423003a15b54b36abdede9fc0dd5c49', 'request_id': 'req-2efffc30-71ff-4460-ad5e-b165a44954cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.568462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] The token cache is being invalidate because project a423003a15b54b36abdede9fc0dd5c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:29.569197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.570166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.576061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 951/1917] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:29 2022] DELETE /identity/v3/projects/a423003a15b54b36abdede9fc0dd5c49 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:29.577363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:29.578783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:29.609293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a506d4910a0452e944f56515d1ea23d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.610604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a506d4910a0452e944f56515d1ea23d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.641123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a506d4910a0452e944f56515d1ea23d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.653588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a506d4910a0452e944f56515d1ea23d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.654723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a506d4910a0452e944f56515d1ea23d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.687497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 967/1918] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:29 2022] DELETE /identity/v3/users/4a506d4910a0452e944f56515d1ea23d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:29.688281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efffc30-71ff-4460-ad5e-b165a44954cf admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:29.797152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:29.837352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:29.838981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:29.839587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:29.840036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] PATH_INFO: `/v3/users/b4fad1a90f7e4690a1226b335a95710b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:29.842853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4fad1a90f7e4690a1226b335a95710b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:29.845605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:29.933869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4fad1a90f7e4690a1226b335a95710b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.947328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4fad1a90f7e4690a1226b335a95710b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.948465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4fad1a90f7e4690a1226b335a95710b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.949516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4fad1a90f7e4690a1226b335a95710b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.968033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-64f41ff1-6b75-4b29-b604-472e952a0637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4fad1a90f7e4690a1226b335a95710b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:29.989964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 952/1919] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:29 2022] DELETE /identity/v3/users/b4fad1a90f7e4690a1226b335a95710b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:30.427445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:30.429035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:30.443569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bde3e6-5a22-4f0f-a8ba-f9b65dc6d557 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:30.474952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:30.532180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:30.533005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:30.533352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:30.533630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] PATH_INFO: `/v3/projects/71852375d53a4482ab9e0e81ca2f2abb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:30.537181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=71852375d53a4482ab9e0e81ca2f2abb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:30.541824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:30.639596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71852375d53a4482ab9e0e81ca2f2abb', 'request_id': 'req-40e8af5d-f048-4d2c-92df-b194922801ba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:30.656192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71852375d53a4482ab9e0e81ca2f2abb', 'request_id': 'req-40e8af5d-f048-4d2c-92df-b194922801ba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:30.671060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71852375d53a4482ab9e0e81ca2f2abb', 'request_id': 'req-40e8af5d-f048-4d2c-92df-b194922801ba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:30.700317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] The token cache is being invalidate because project 71852375d53a4482ab9e0e81ca2f2abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:30.700759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:30.705140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:30.711993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 968/1920] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:30 2022] DELETE /identity/v3/projects/71852375d53a4482ab9e0e81ca2f2abb => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:30.714899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b08e3-5c38-401d-b031-b019904cdf46 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:30.715406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b08e3-5c38-401d-b031-b019904cdf46 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:30.715662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b08e3-5c38-401d-b031-b019904cdf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:30.773929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1d6b08e3-5c38-401d-b031-b019904cdf46 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:30.774748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1d6b08e3-5c38-401d-b031-b019904cdf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:30.781285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:30.782930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:30.929562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 953/1921] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:30.948191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e095c1-8485-48fd-a081-46659184f767 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:30.948556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e095c1-8485-48fd-a081-46659184f767 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:30.948841 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e095c1-8485-48fd-a081-46659184f767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:30.979925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8af5d-f048-4d2c-92df-b194922801ba admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:30.992185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:30.993999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:30.996132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:30.996608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] PATH_INFO: `/v3/projects/1182be89159c45588a2baefa10099149` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:30.999095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=1182be89159c45588a2baefa10099149)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.001846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-20e095c1-8485-48fd-a081-46659184f767 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:31.002861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.003251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-20e095c1-8485-48fd-a081-46659184f767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.015398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 954/1922] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:31.040047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e095c1-8485-48fd-a081-46659184f767 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.055643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e095c1-8485-48fd-a081-46659184f767 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.077094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1182be89159c45588a2baefa10099149', 'request_id': 'req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:31.085926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1182be89159c45588a2baefa10099149', 'request_id': 'req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:31.087428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:31.094115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1182be89159c45588a2baefa10099149', 'request_id': 'req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:31.107435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] The token cache is being invalidate because project 1182be89159c45588a2baefa10099149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:31.108391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:31.109878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-6e1d43cc-a95b-47dc-afa1-ad70290c16ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:31.112407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 969/1923] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:30 2022] DELETE /identity/v3/projects/1182be89159c45588a2baefa10099149 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:31.118000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441a8f9-ebcd-42d3-9bca-cc290466d1c7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.118940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441a8f9-ebcd-42d3-9bca-cc290466d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.119220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441a8f9-ebcd-42d3-9bca-cc290466d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.137518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.138377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.138687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.138950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.139650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:31.140110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.141805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.148450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5441a8f9-ebcd-42d3-9bca-cc290466d1c7 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:31.149902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5441a8f9-ebcd-42d3-9bca-cc290466d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.158357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 970/1924] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:31.166868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:31.167522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.168021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.168283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.176942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 955/1925] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:31.177690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.179351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.195141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:31.199106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.215961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e952-943f-4e39-a50a-d7b70c6ad2c9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.222450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 971/1926] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:31.227663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.238132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.239061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.239967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.240746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.242808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.250872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.254329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98138ca4-dbae-460a-8b65-e4dc03901cc6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.267712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.268690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.269111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.269674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.272203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:31.274215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.276059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.309616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 972/1927] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:31.311619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.327082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 956/1928] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:31.328090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.340848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31c011-fcc5-458f-89e7-8e95016f13a6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.423146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e56a46-8b30-49a9-98b8-adfc026518d0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.430797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.432469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.432697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.433167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.433267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.433729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.434113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.434222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.434558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.435373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.437541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.437891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.477068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 973/1929] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:31.477280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 957/1930] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:31.479809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.480262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.491745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366ee9c-47d5-41bd-8c2a-caf6a9d6a570 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.588150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109218c3-4b22-4aeb-ad4e-ac0a390955fa admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.593633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.594369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.594803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.595287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.596246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.598297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.612049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.612448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.612705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.613001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.614627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.616609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.631725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 958/1931] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:31.635766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.640423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 974/1932] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:31.644441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.652324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b65ce4-9eb2-4569-a321-a98dcdbd53ee admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.659698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fec3c4-0d6a-40ea-9cb3-8903748f3141 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.659977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.660820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.661131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.661507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.662292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.664107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.665159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.667493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.667781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.668057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.668814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.671023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.684380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 959/1933] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:31.685870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 975/1934] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:31.686854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.689837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.699922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3727328-8242-4c7d-9984-8bbe5c8a20d3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.702700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61c5e7-2b84-4414-8a03-160fa2012cf7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.707475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.709169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.709169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.709517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.709723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] PATH_INFO: `/v3/projects/c51d4c4aa3fc496eb3ea55eaf5949f12/users/c92bfcccb95c45f3bf1ca5911f0f85f7/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.709890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.709890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.709890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.710715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.713271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.726158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 976/1935] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:31.730718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.735007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c51d4c4aa3fc496eb3ea55eaf5949f12, user_id=c92bfcccb95c45f3bf1ca5911f0f85f7, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.737746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa31d90-c8fb-4fbb-b7cc-58ecb3594b0a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.741064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29bc7f-26c4-483a-9941-605998aab9ee admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.746363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.747078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.747364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.747646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] PATH_INFO: `/v3/projects/15031d3278944e9793217b5ea662631f/users/302bacd46a824ecabb9a8aa3993d62b3/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.751818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 960/1936] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:31 2022] PUT /identity/v3/projects/c51d4c4aa3fc496eb3ea55eaf5949f12/users/c92bfcccb95c45f3bf1ca5911f0f85f7/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:31.755762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:31.763620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.763901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.764177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.778634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15031d3278944e9793217b5ea662631f, user_id=302bacd46a824ecabb9a8aa3993d62b3, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.782187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae962dd-f816-49fa-bf62-c18b444ba697 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.798093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 977/1937] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:31 2022] PUT /identity/v3/projects/15031d3278944e9793217b5ea662631f/users/302bacd46a824ecabb9a8aa3993d62b3/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:31.798621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] MFA Rules not processed for user `c92bfcccb95c45f3bf1ca5911f0f85f7`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:31.799900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.811299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.811592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.811870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.813752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:31.841404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] MFA Rules not processed for user `302bacd46a824ecabb9a8aa3993d62b3`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:31.842627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.872759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 961/1938] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:31.877207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.917027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 978/1939] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:31.925541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:31.927275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:31.944130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76115b8a-f2fb-49a3-89ce-86d650732b83 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:31.950461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:31.951270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:31.951667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:31.952040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:31.952849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:31.954936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:31.988289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 962/1940] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:31.991997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.000543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc4438-b91c-4429-b258-4d9a2b29fa32 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.072964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.073482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.073772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.074053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.099991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e5b55-bec5-4d54-bb59-4c5290930015 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.107113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.107762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.108047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.108858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.109712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.112851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.139436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:32.142816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 963/1941] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:31 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:32.143496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.159274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.160896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.161750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe30f2-b961-448d-99e7-5ef90d2a9af9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.167566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.168629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.168917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.169220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.169891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.171464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 979/1942] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:32.176938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.194555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 964/1943] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:32.194738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.196154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.209749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c5cac-f222-4cbd-9619-57d233159ac5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.216509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.217196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.217478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.217757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] PATH_INFO: `/v3/projects/e3f848d486f94bd484e0c37c48cce06e/users/73660494bf2a4dd6a96d6adb423941fc/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.235055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2754ed79-66dd-4c24-ad61-4fe6d0dacf2f service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.241738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.242256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f848d486f94bd484e0c37c48cce06e, user_id=73660494bf2a4dd6a96d6adb423941fc, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.242392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.242706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.242982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] PATH_INFO: `/v3/users/302bacd46a824ecabb9a8aa3993d62b3` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.244394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=302bacd46a824ecabb9a8aa3993d62b3)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.245343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.246418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.249201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:32.267337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 965/1944] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:32 2022] PUT /identity/v3/projects/e3f848d486f94bd484e0c37c48cce06e/users/73660494bf2a4dd6a96d6adb423941fc/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:32.271667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.310862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302bacd46a824ecabb9a8aa3993d62b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:32.311578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302bacd46a824ecabb9a8aa3993d62b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:32.320164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302bacd46a824ecabb9a8aa3993d62b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:32.330605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302bacd46a824ecabb9a8aa3993d62b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:32.331345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a14e165d-742c-4762-bc2d-d92df9efaff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302bacd46a824ecabb9a8aa3993d62b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:32.349953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26dd1d4-4766-4c54-bf2d-314001e46fd1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.352662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 980/1945] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:32 2022] DELETE /identity/v3/users/302bacd46a824ecabb9a8aa3993d62b3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:32.355293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.355923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.356200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.356510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.357203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.357454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.357830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.358212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.358826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.372830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 966/1946] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:32.374054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.375849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:32.378654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:32.379897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:32.389558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2af701-7e15-40b2-86bf-65c5a13b7cb5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.444869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 981/1947] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:32.446643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.447820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.448104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.448406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] PATH_INFO: `/v3/projects/e3f848d486f94bd484e0c37c48cce06e/users/73660494bf2a4dd6a96d6adb423941fc/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.450537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3f848d486f94bd484e0c37c48cce06e, user_id=73660494bf2a4dd6a96d6adb423941fc, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.453542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd3e3-a3ee-4348-82eb-4a530e95d10a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.463232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.464632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:32.464885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 967/1948] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:32 2022] PUT /identity/v3/projects/e3f848d486f94bd484e0c37c48cce06e/users/73660494bf2a4dd6a96d6adb423941fc/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:32.475167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da680-4251-406f-b224-7b197ebb319f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.475464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da680-4251-406f-b224-7b197ebb319f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.475746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18da680-4251-406f-b224-7b197ebb319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.505292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a18da680-4251-406f-b224-7b197ebb319f None None] MFA Rules not processed for user `73660494bf2a4dd6a96d6adb423941fc`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:32.506507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a18da680-4251-406f-b224-7b197ebb319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:32.525966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ac0f2-538c-496f-85d4-c2f475e15d0c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.538060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:32.580143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 968/1949] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:32 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:32.593259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da680-4251-406f-b224-7b197ebb319f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:32.595264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a18da680-4251-406f-b224-7b197ebb319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:32.596046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.596723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.597568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.597925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.616611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:32.679885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18da680-4251-406f-b224-7b197ebb319f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:32.690202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:32.691351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:32.691801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:32.691940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:32.698278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.700787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.706954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:32.708439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730abc31-df12-45b6-9d37-1ef658fd6958 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:32.715033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 982/1950] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:32.724607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 969/1951] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:33.060779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.064451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.096623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d589c-bb50-4915-b0ad-ed3c8fb39ad6 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.105860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.106750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.107183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.107804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] PATH_INFO: `/v3/projects/15031d3278944e9793217b5ea662631f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.109271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=15031d3278944e9793217b5ea662631f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.111545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.174987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15031d3278944e9793217b5ea662631f', 'request_id': 'req-c023acab-2b40-48f5-8ec7-59d629e261ef'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.189918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15031d3278944e9793217b5ea662631f', 'request_id': 'req-c023acab-2b40-48f5-8ec7-59d629e261ef'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.205682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15031d3278944e9793217b5ea662631f', 'request_id': 'req-c023acab-2b40-48f5-8ec7-59d629e261ef'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.224849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] The token cache is being invalidate because project 15031d3278944e9793217b5ea662631f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:33.224849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.225960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.228361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 983/1952] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:33 2022] DELETE /identity/v3/projects/15031d3278944e9793217b5ea662631f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:33.234144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6f98d-3ea1-4787-a16b-fd100655c9ee None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.234627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6f98d-3ea1-4787-a16b-fd100655c9ee None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.234822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6f98d-3ea1-4787-a16b-fd100655c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.286027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-41b6f98d-3ea1-4787-a16b-fd100655c9ee None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:33.290440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-41b6f98d-3ea1-4787-a16b-fd100655c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.383643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.386327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.447672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 970/1953] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:33.449478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.449772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.450069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.473621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:33.474910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.483002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:33.492551 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 971/1954] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:33.496668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.504530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023acab-2b40-48f5-8ec7-59d629e261ef admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.510007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776dd92-4dd0-4899-a257-77c7653db125 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.510329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.511028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.512051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.512051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] PATH_INFO: `/v3/users/c92bfcccb95c45f3bf1ca5911f0f85f7` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.512860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92bfcccb95c45f3bf1ca5911f0f85f7)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.514798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.526882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.528039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.528592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.529420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.530363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:33.532864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.535080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.557342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 972/1955] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:33.561833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.573548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a73b04-4911-4c4d-94b9-7c9a76c8222c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.579255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.579894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.580182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.580498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.581239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.583985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.585923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92bfcccb95c45f3bf1ca5911f0f85f7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.588467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92bfcccb95c45f3bf1ca5911f0f85f7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.615323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92bfcccb95c45f3bf1ca5911f0f85f7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.623594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 973/1956] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:33.623808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92bfcccb95c45f3bf1ca5911f0f85f7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.624526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ce8208a-41bc-4b52-a965-5637b235504d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92bfcccb95c45f3bf1ca5911f0f85f7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:33.628533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.630318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.658653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 984/1957] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:33 2022] DELETE /identity/v3/users/c92bfcccb95c45f3bf1ca5911f0f85f7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:55:33.660145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.660312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.660798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.680184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:33.681753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.707531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35a4d8-1d8b-421f-b73a-799fa8165a75 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.725117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:33.746828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 985/1958] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:33.753619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.755030 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:33.768260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.768917 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.769204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.769481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.770148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.772090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.799485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 974/1959] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:33.800833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.810805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde853d-e0d8-430c-a0dc-22379a85c837 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.815383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.815996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.816296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.816594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.817294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.817876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3edcd8-99e9-4bb8-a761-2c4a22fa7820 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.818906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.822338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.822960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.823261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.823512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.829813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 975/1960] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:33.833715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.833863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.835299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.844396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c60a35-de49-42be-9c24-756b3c9fa82c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.846521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 986/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:55:33.849420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.851099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.851099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.851099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.852624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.854645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b733e7f-46a8-4762-9c83-9009632ab83a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.865753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 976/1962] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:33.870871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:33.881310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0d392-2f9f-45c2-a111-7728514b2a3b service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:33.888084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:33.888809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.889098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.889473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] PATH_INFO: `/v3/projects/a7a72085302141148a3ad70cceccf53f/users/33a6d1b1f8e44670bbd4e603823be55c/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.898946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:33.911216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a72085302141148a3ad70cceccf53f, user_id=33a6d1b1f8e44670bbd4e603823be55c, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:33.914051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:33.930615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 987/1963] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:33 2022] PUT /identity/v3/projects/a7a72085302141148a3ad70cceccf53f/users/33a6d1b1f8e44670bbd4e603823be55c/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:33.941408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:33.941675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:33.941955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:33.973147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] MFA Rules not processed for user `33a6d1b1f8e44670bbd4e603823be55c`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:33.974433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:34.046784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 977/1964] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:34.059432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:34.061361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:34.141418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c481f9a-11b8-4a5c-9ce8-d67ab81be67d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:34.145369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:34.147348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:34.147348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:34.147348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:34.159642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:34.161698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:34.173268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 988/1965] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:34.237720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:34.239391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:34.330418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acbec52-ca75-4932-b064-3c18a717b044 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:34.335385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:34.336158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:34.336426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:34.336837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] PATH_INFO: `/v3/projects/c51d4c4aa3fc496eb3ea55eaf5949f12` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:34.338298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c51d4c4aa3fc496eb3ea55eaf5949f12)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:34.340269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:34.365489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:34.404293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c51d4c4aa3fc496eb3ea55eaf5949f12', 'request_id': 'req-11c2e0f0-575c-400a-a961-17aed458d8ae'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.413451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c51d4c4aa3fc496eb3ea55eaf5949f12', 'request_id': 'req-11c2e0f0-575c-400a-a961-17aed458d8ae'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.420839 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c51d4c4aa3fc496eb3ea55eaf5949f12', 'request_id': 'req-11c2e0f0-575c-400a-a961-17aed458d8ae'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.435251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] The token cache is being invalidate because project c51d4c4aa3fc496eb3ea55eaf5949f12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:34.435569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.436296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.438221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 978/1966] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:34 2022] DELETE /identity/v3/projects/c51d4c4aa3fc496eb3ea55eaf5949f12 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:34.442572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:34.455793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7758ef-d35f-46ad-a7f5-90ce3d559d12 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:34.548576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:34.549958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:34.550686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:34.551118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] PATH_INFO: `/v3/users/73660494bf2a4dd6a96d6adb423941fc` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:34.552368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73660494bf2a4dd6a96d6adb423941fc)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:34.554318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:34.615233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73660494bf2a4dd6a96d6adb423941fc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.616764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73660494bf2a4dd6a96d6adb423941fc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.629193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73660494bf2a4dd6a96d6adb423941fc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.638295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73660494bf2a4dd6a96d6adb423941fc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.639794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c00ff405-e9de-48b6-90f8-4e07b9161823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73660494bf2a4dd6a96d6adb423941fc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:34.666659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 989/1967] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:34 2022] DELETE /identity/v3/users/73660494bf2a4dd6a96d6adb423941fc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:35.062921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.064536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:35.082008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e0f0-575c-400a-a961-17aed458d8ae admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.146244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.147021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.147411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.147844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] PATH_INFO: `/v3/projects/e3f848d486f94bd484e0c37c48cce06e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.149576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3f848d486f94bd484e0c37c48cce06e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.152309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.206046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3f848d486f94bd484e0c37c48cce06e', 'request_id': 'req-92a90cb3-540a-4072-b4a9-21a43c00b522'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:35.214516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3f848d486f94bd484e0c37c48cce06e', 'request_id': 'req-92a90cb3-540a-4072-b4a9-21a43c00b522'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:35.221036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3f848d486f94bd484e0c37c48cce06e', 'request_id': 'req-92a90cb3-540a-4072-b4a9-21a43c00b522'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:35.235761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] The token cache is being invalidate because project e3f848d486f94bd484e0c37c48cce06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:35.236062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:35.236750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-92a90cb3-540a-4072-b4a9-21a43c00b522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:35.238480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 979/1968] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:35 2022] DELETE /identity/v3/projects/e3f848d486f94bd484e0c37c48cce06e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:35.243382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.244916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.244916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.276888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:35.278300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:35.289127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:35.351367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 990/1969] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:35.359635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.359926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.360205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.380189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:35.381416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:35.389997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 980/1970] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:35.394345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.404167 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f220c-dc7a-4037-9203-898d5d21869f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.408806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.409414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.409694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.409966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.410662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:35.410960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.412574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.425873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 991/1971] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:35.430554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.441035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f6f6a-1a9a-4d5a-b3a1-a3e4ce0b185f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.445906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.447150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.447871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.447871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.448359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.450255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.484643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 981/1972] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:35 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:35.489528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.500201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db6f30-b3e8-4f8b-a821-778ed2a7dbe8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.559957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.560655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.560883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.561158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.561856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.563772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.592172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 992/1973] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:35 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:35.598028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.612760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87442c9f-1263-48c1-8fce-3ffeaa877dba admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.621807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.622869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.623373 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.623868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.625211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.628541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.632382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:35.646851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 982/1974] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:35.652104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.662320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ed4c25-e6d9-4bcf-9508-ff48345374eb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.667968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.668616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.669071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.669348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.670106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.671733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488abc66-5b40-4c37-a898-034f3d82d98b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.684452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 993/1975] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:35.689061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.699886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c091fd-62ed-4313-a2e5-b27e0124dfcf admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.705036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.705760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.706130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.706587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] PATH_INFO: `/v3/projects/cdc7e4ea5e584278ac388a18e02e8784/users/eadfb0f605a24b7baf42225d14df4e6a/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.725562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc7e4ea5e584278ac388a18e02e8784, user_id=eadfb0f605a24b7baf42225d14df4e6a, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.729567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.746876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 983/1976] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:35 2022] PUT /identity/v3/projects/cdc7e4ea5e584278ac388a18e02e8784/users/eadfb0f605a24b7baf42225d14df4e6a/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:35.752222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512c13d-5307-45ce-afc5-2582d19b7724 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.752638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512c13d-5307-45ce-afc5-2582d19b7724 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.752995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512c13d-5307-45ce-afc5-2582d19b7724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.772938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-1512c13d-5307-45ce-afc5-2582d19b7724 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:35.774193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-1512c13d-5307-45ce-afc5-2582d19b7724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:35.858805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 994/1977] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:35.870164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:35.872286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:35.934081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c73e51-522d-4734-baba-98beba265e30 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:35.937931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:35.938651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:35.938939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:35.939217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:35.949804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:35.951346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:35.961768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 984/1978] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:36.788098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:36.788651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:36.788740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:36.818873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] MFA Rules not processed for user `eadfb0f605a24b7baf42225d14df4e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:36.820298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:36.825510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:36.908076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 995/1979] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:36.912517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:36.923092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab79c146-91a5-4d6a-927e-ada8c02fc7b1 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:36.928456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:36.929076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:36.929385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:36.929656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:36.930325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:36.932884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:36.966450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 985/1980] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:36 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:55:36.967969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:36.979397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b834-301a-4efc-b9bc-a6414350fc88 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:37.042854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:37.043494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:37.043778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:37.044058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:37.044786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:37.046722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:37.070359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 996/1981] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:36 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:37.075082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:37.085669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e98ad1-b063-4a6c-bace-c14f160e999b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:37.090679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:37.091305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:37.091585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:37.091858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:37.092584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:37.094222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:37.105601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 986/1982] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:37.110327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:37.122132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d0add-bc1a-4f5e-8440-a0844a581ec4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:37.126858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:37.127571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:37.127857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:37.128138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] PATH_INFO: `/v3/projects/26626ccf37e6487c885ff8d1ddd67251/users/7f5f28b090554512b45a070bde9e97c9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:37.148005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26626ccf37e6487c885ff8d1ddd67251, user_id=7f5f28b090554512b45a070bde9e97c9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:37.151184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:37.167012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 997/1983] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:37 2022] PUT /identity/v3/projects/26626ccf37e6487c885ff8d1ddd67251/users/7f5f28b090554512b45a070bde9e97c9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:37.173259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:37.185541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f7e38-2e09-489f-9824-73c5b4a1ca24 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:37.246504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:37.247119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:37.247399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:37.247684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:37.248389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:37.249984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdffdbb-2d03-4676-8acb-cddb20cdc45f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:37.261420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 987/1984] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:37.266140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:37.277620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140f5baa-5543-43a8-9b3d-71702a428830 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:37.282314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:37.282996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:37.283284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:37.283560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] PATH_INFO: `/v3/projects/26626ccf37e6487c885ff8d1ddd67251/users/7f5f28b090554512b45a070bde9e97c9/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:37.286409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=26626ccf37e6487c885ff8d1ddd67251, user_id=7f5f28b090554512b45a070bde9e97c9, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:37.289172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:37.301955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 998/1985] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:37 2022] PUT /identity/v3/projects/26626ccf37e6487c885ff8d1ddd67251/users/7f5f28b090554512b45a070bde9e97c9/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:38.055180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:38.056102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:38.056102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:38.092718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] MFA Rules not processed for user `7f5f28b090554512b45a070bde9e97c9`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:38.093998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:38.132965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:38.188695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 988/1986] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:38.200592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:38.203025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:38.279290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82543c7c-e7ed-4059-b428-3a37ce4e65bc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:38.282899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:38.283619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:38.283912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:38.284165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:38.293886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:38.347007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:38.348491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:38.358220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 999/1987] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:44.032810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:44.034577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:44.048212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16137f1-33aa-4939-9c9a-cd4554615dbc None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:44.052683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:44.053454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:44.053853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:44.054245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:44.065052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:44.066597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77da48-bbb3-4b67-aeb0-45d20e8f5649 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:44.078520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 989/1988] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:46.042570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:46.044061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:46.119681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388ea2e-5b4d-4228-bb6b-5c1298369fc5 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:46.124606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:46.125403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:46.125695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:46.126122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] PATH_INFO: `/v3/users/33a6d1b1f8e44670bbd4e603823be55c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:46.127489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33a6d1b1f8e44670bbd4e603823be55c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:46.129982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:46.203270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33a6d1b1f8e44670bbd4e603823be55c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.203968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33a6d1b1f8e44670bbd4e603823be55c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.212688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33a6d1b1f8e44670bbd4e603823be55c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.222436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33a6d1b1f8e44670bbd4e603823be55c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.223168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33a6d1b1f8e44670bbd4e603823be55c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.253245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1000/1989] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:55:46 2022] DELETE /identity/v3/users/33a6d1b1f8e44670bbd4e603823be55c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:46.259517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:46.259980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:46.260100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:46.286475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:46.287782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:46.355729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 990/1990] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:46.365855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:46.367557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:46.408542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:46.437361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e22bf-2af8-4bb5-9270-e8e255f38ed4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:46.442229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:46.442872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:46.443699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:46.444000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:46.454824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:46.456270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e696-5645-4c18-8f68-b4470cc7b76a service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:46.467431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1001/1991] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:46.791573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:46.793147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:46.808708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b1a7b-9aaf-4397-acc7-ad8b6fee9b85 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:46.814042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:46.814949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:46.815248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:46.815530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] PATH_INFO: `/v3/projects/a7a72085302141148a3ad70cceccf53f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:46.816786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a72085302141148a3ad70cceccf53f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:46.818727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:46.882751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a72085302141148a3ad70cceccf53f', 'request_id': 'req-b6a649b8-aadc-40cb-b076-080da55532e3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.893206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a72085302141148a3ad70cceccf53f', 'request_id': 'req-b6a649b8-aadc-40cb-b076-080da55532e3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.901875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a72085302141148a3ad70cceccf53f', 'request_id': 'req-b6a649b8-aadc-40cb-b076-080da55532e3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.922151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] The token cache is being invalidate because project a7a72085302141148a3ad70cceccf53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:55:46.922762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.923810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b6a649b8-aadc-40cb-b076-080da55532e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:55:46.928270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 991/1992] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:55:46 2022] DELETE /identity/v3/projects/a7a72085302141148a3ad70cceccf53f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:46.935906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:46.936312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:46.937098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:46.959904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:46.961400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:47.058603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1002/1993] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:47.066294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.066611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.066849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.090464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:47.091698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:47.101776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 992/1994] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:55:47.107571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.118502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9970b5d7-ab91-4df0-bae8-f1958a781a6b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.123613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.124242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.124714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.125185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.125890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:55:47.126196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.127838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.143195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1003/1995] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:55:47.148372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.159017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae7449a-ae9f-4fa9-a684-6a5ed72c4a16 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.164125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.164792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.165090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.165365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.166067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.167934 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.213768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 993/1996] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:55:47 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:47.215586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.232109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a36d3-33c4-4e85-950f-1af081cf5fea admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.347453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.348603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.349479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.350039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.351515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.354743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.384502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1004/1997] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:55:47 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:55:47.386370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.402130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a34d68-7990-4638-a8ea-a5562c842a99 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.408203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.409131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.421085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.421596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.422682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.427683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.433686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:47.452785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 994/1998] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:55:47.453432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.476151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d440b9-9503-44e3-8ee5-eff85f5d0fe6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.485871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.486844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.487175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.487640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.488655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.491286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41e4a9-38d0-43c7-a5af-3bb138b110e1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.512237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1005/1999] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:55:47.516506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.542269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e43d30-05f7-4948-8a62-db973819b1e1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.550048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.551588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.552376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.552732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] PATH_INFO: `/v3/projects/22e6f064a6b34a67bd935bd3923e6829/users/828614d1dc6840a19f7f2d970976ae5d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.577320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e6f064a6b34a67bd935bd3923e6829, user_id=828614d1dc6840a19f7f2d970976ae5d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.580210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.596753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 995/2000] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:55:47 2022] PUT /identity/v3/projects/22e6f064a6b34a67bd935bd3923e6829/users/828614d1dc6840a19f7f2d970976ae5d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:55:47.608083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.608445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.608726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.641100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] MFA Rules not processed for user `828614d1dc6840a19f7f2d970976ae5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:55:47.642318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:47.676312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:47.732441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1006/2001] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:55:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:55:47.745070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:55:47.747066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:55:47.817296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b0aca-9efc-4fb0-9510-c6e8c03c0d77 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:55:47.821381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:55:47.822517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:55:47.822869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:55:47.823283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:55:47.833495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:55:47.834959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98672070-34ab-4882-83e7-9ce44d49d4d7 service glance] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:55:47.845903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 996/2002] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:55:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:55:57.720666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=14 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:55:57.841264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:03.261668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:03.263429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:03.331513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a15f16-c76f-4be3-ace2-3b64d5485801 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:03.336074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:03.336743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:03.337023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:03.337295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] PATH_INFO: `/v3/users/9906d7d342664ae6b1e4a3f8df5b5662` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:03.338439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9906d7d342664ae6b1e4a3f8df5b5662)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:03.340710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:03.397041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9906d7d342664ae6b1e4a3f8df5b5662'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:03.397697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9906d7d342664ae6b1e4a3f8df5b5662'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:03.406072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9906d7d342664ae6b1e4a3f8df5b5662'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:03.415037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9906d7d342664ae6b1e4a3f8df5b5662'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:03.415958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9906d7d342664ae6b1e4a3f8df5b5662'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:03.436598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1007/2003] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:03 2022] DELETE /identity/v3/users/9906d7d342664ae6b1e4a3f8df5b5662 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:03.441141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:03.441637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:03.441740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:03.462265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:03.463470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:03.525811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 997/2004] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:03.535699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:03.537157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:03.598285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a4bb2d-2b6c-4a10-a7bd-b3897b795e97 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:03.602391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:03.603042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:03.603327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:03.603631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:03.627509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:03.629003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f98e14-1f03-4607-abdd-6e52a566ee2c service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:03.639672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1008/2005] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:03.934674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:03.936224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:03.962641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c7bab-222f-4669-8b1d-60c4dd8d963e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:03.970128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:03.970846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:03.971201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:03.971519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] PATH_INFO: `/v3/projects/96efe0a69fc44794b2c5358528fa63b2` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:03.972741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96efe0a69fc44794b2c5358528fa63b2)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:03.974673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.031402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96efe0a69fc44794b2c5358528fa63b2', 'request_id': 'req-3883eaf2-8d71-40d6-8685-98ece3ea6e65'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:04.040080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96efe0a69fc44794b2c5358528fa63b2', 'request_id': 'req-3883eaf2-8d71-40d6-8685-98ece3ea6e65'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:04.047381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96efe0a69fc44794b2c5358528fa63b2', 'request_id': 'req-3883eaf2-8d71-40d6-8685-98ece3ea6e65'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:04.063661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] The token cache is being invalidate because project 96efe0a69fc44794b2c5358528fa63b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:04.063962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:04.064652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3883eaf2-8d71-40d6-8685-98ece3ea6e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:04.066224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 998/2006] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:03 2022] DELETE /identity/v3/projects/96efe0a69fc44794b2c5358528fa63b2 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:04.071937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.071937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.071937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.104867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:04.106154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:04.139610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:04.207654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1009/2007] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:04.217249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.217533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.217816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.241487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:04.242727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:04.257052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 999/2008] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:04.258997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.270893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32e21a3-8925-45eb-a19c-a32fe3b77422 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.276512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.277273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.277552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.277824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.278520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:04.278842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.280460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.298879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1010/2009] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:04.300090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.310904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7abc3-3693-43cd-81ee-f0394fd83173 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.315580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.316239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.316554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.316857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.317527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.319429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.351022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1000/2010] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:04 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:04.353121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.365552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30e213-3bc2-4e78-bdd2-0d3f8f3627ca admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.423356 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.423983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.424266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.424576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.425249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.427427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.449515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1011/2011] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:04 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:04.453776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.464834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9d91d-4028-4b1c-9095-90ed5e497380 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.469613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.470371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.470519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.470809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.471462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.473118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.488384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1001/2012] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:04.489564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.501122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c559d35-d7c7-4efb-811a-8bbd7198ae8c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.506073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.511811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.512192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.512515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.513284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.514938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20459591-8fce-45b1-b9a8-abf0e19ceecb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.527335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1012/2013] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:04.531698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.541892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb26afd-1fea-453c-919f-8804a6bde6b2 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.546498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.547162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.547445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.547902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] PATH_INFO: `/v3/projects/3b320ff74c544841a3917d8a3628c225/users/c175b23fd9b04701b62bd7a49f7e2ee6/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.564083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b320ff74c544841a3917d8a3628c225, user_id=c175b23fd9b04701b62bd7a49f7e2ee6, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.566885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.580252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1002/2014] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:04 2022] PUT /identity/v3/projects/3b320ff74c544841a3917d8a3628c225/users/c175b23fd9b04701b62bd7a49f7e2ee6/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:04.590573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35daa2-c1ef-4548-86f6-27a62096902e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.590877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35daa2-c1ef-4548-86f6-27a62096902e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.591178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35daa2-c1ef-4548-86f6-27a62096902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.617857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0d35daa2-c1ef-4548-86f6-27a62096902e None None] MFA Rules not processed for user `c175b23fd9b04701b62bd7a49f7e2ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:04.619057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0d35daa2-c1ef-4548-86f6-27a62096902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:04.689691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1013/2015] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:04 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:04.702932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:04.704702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:04.726957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:04.766913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b0983-58a9-4a0e-b64e-212e0de78a5c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:04.770503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:04.771113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:04.771392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:04.771670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:04.779954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:04.781419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763127c6-e355-46b3-8326-13884953bbdf service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:04.789981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1003/2016] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:06.074418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:06.074960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:06.074960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:06.076318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1014/2017] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:56:06.103004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:06.103519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:06.103635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:06.124059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:06.125605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:06.206995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1004/2018] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:56:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:06.219267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:06.221087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:06.241817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761cc13b-34a6-4b77-a8da-4b54da87635e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:06.254703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:06.255663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:06.255953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:06.256461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:06.268123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:06.271868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:06.293575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1015/2019] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:06.766045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:06.767607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:06.840728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53840249-745e-4d87-88e5-f2ed2e1b862e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:06.845452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:06.846117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:06.846399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:06.846672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] PATH_INFO: `/v3/users/eadfb0f605a24b7baf42225d14df4e6a` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:06.847931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eadfb0f605a24b7baf42225d14df4e6a)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:06.849903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:06.899558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eadfb0f605a24b7baf42225d14df4e6a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:06.907745 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eadfb0f605a24b7baf42225d14df4e6a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:06.908399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eadfb0f605a24b7baf42225d14df4e6a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:06.909023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eadfb0f605a24b7baf42225d14df4e6a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:06.914825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:06.924808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eadfb0f605a24b7baf42225d14df4e6a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:06.939779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1005/2020] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:06 2022] DELETE /identity/v3/users/eadfb0f605a24b7baf42225d14df4e6a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:07.114692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:07.122927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:07.128184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38535658-a21a-4df5-967b-628bbc485ace service placement] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:07.189104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:07.189763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.190046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.190343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] PATH_INFO: `/v3/projects/cdc7e4ea5e584278ac388a18e02e8784` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.191540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc7e4ea5e584278ac388a18e02e8784)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:07.193500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:07.240768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc7e4ea5e584278ac388a18e02e8784', 'request_id': 'req-45ab9ebb-380a-4800-bd19-7f96ef74f222'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.249052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc7e4ea5e584278ac388a18e02e8784', 'request_id': 'req-45ab9ebb-380a-4800-bd19-7f96ef74f222'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.254845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc7e4ea5e584278ac388a18e02e8784', 'request_id': 'req-45ab9ebb-380a-4800-bd19-7f96ef74f222'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.268061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] The token cache is being invalidate because project cdc7e4ea5e584278ac388a18e02e8784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:07.268330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.269078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.270705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1016/2021] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:07 2022] DELETE /identity/v3/projects/cdc7e4ea5e584278ac388a18e02e8784 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:07.274739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:07.277617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:07.291169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71768cbd-9038-4e72-83bb-bdd45e193f57 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:07.352813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:07.353471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.353756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.354034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] PATH_INFO: `/v3/users/7f5f28b090554512b45a070bde9e97c9` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.356507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5f28b090554512b45a070bde9e97c9)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:07.358449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:07.405870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5f28b090554512b45a070bde9e97c9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.414515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5f28b090554512b45a070bde9e97c9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.415169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5f28b090554512b45a070bde9e97c9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.415867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5f28b090554512b45a070bde9e97c9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.429063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-fd0bfb0f-50e9-4cae-a37e-cd483139e9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5f28b090554512b45a070bde9e97c9'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.441844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1006/2022] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:07 2022] DELETE /identity/v3/users/7f5f28b090554512b45a070bde9e97c9 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:07.634052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:07.635646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:07.649696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab9ebb-380a-4800-bd19-7f96ef74f222 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:07.715223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:07.715941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.716224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.716522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] PATH_INFO: `/v3/projects/26626ccf37e6487c885ff8d1ddd67251` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.717700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26626ccf37e6487c885ff8d1ddd67251)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:07.719775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:07.759240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:07.770353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26626ccf37e6487c885ff8d1ddd67251', 'request_id': 'req-743071a2-8c16-45ba-b391-def2331220b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.779758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26626ccf37e6487c885ff8d1ddd67251', 'request_id': 'req-743071a2-8c16-45ba-b391-def2331220b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.785662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26626ccf37e6487c885ff8d1ddd67251', 'request_id': 'req-743071a2-8c16-45ba-b391-def2331220b3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.798179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] The token cache is being invalidate because project 26626ccf37e6487c885ff8d1ddd67251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:07.798571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.799245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-743071a2-8c16-45ba-b391-def2331220b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:07.801080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1017/2023] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:07 2022] DELETE /identity/v3/projects/26626ccf37e6487c885ff8d1ddd67251 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:07.806695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.806980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.807284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.827147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:07.828415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:07.881469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:07.899482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1007/2024] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:07.908879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.909161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.909433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.930033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:07.931298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:07.944077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1018/2025] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:07.945319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:07.956625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5840510-58da-44ec-b373-eb85e3ac2c46 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:07.961613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:07.962308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.962800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:07.963061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:07.964576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:07.964969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:07.966800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:07.980383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1008/2026] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:07.983559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:07.993359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc464c-6197-452f-a1ac-414d382f3153 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:07.998916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:07.999534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:07.999814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.000087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.000792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.002697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.031482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1019/2027] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:07 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:08.035609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.046269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3455bc71-bfba-4a1b-833a-db7a3801b242 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.107324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.107844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.108127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.108438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.109110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.111033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.133878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1009/2028] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:08 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:08.138747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.148969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba73db8-f143-48d5-82e1-f1b901e763d1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.154393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.155010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.155289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.155560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.156243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.158169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.170163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1020/2029] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:08.174939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.185834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0076a9c-6e55-4807-9507-1c2e7d904a51 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.190873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.191539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.191796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.192072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.192808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.194463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a920e53-4e2f-40bc-be12-1e1bbdf659ce admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.206644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1010/2030] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:08.211366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.222689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ec25-1192-401b-a990-1efc92c17a2b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.227396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.228053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.228332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.228632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] PATH_INFO: `/v3/projects/ef59358a76914143ab89248f8297320e/users/2aae7da8b22b4804865d92e0b1d16292/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.246800 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef59358a76914143ab89248f8297320e, user_id=2aae7da8b22b4804865d92e0b1d16292, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.249478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.265439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1021/2031] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:08 2022] PUT /identity/v3/projects/ef59358a76914143ab89248f8297320e/users/2aae7da8b22b4804865d92e0b1d16292/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:08.277498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.278135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.278135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.307957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] MFA Rules not processed for user `2aae7da8b22b4804865d92e0b1d16292`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:08.309217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:08.377930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1011/2032] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:08.390163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.391861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:08.458273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f186fd7-6cda-48b2-93f5-0b6d092d6ddc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.461929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.462535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.462833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.463104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.473758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.475210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22452657-8152-40d1-8b36-d80dfbe2d2fe service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.485681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1022/2033] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:08.506765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.564265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:08.576144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab829771-38d6-4669-9809-89bdb2797174 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.580899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.581548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.581828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.582102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] PATH_INFO: `/v3/users/2aae7da8b22b4804865d92e0b1d16292` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.583267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aae7da8b22b4804865d92e0b1d16292)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.585238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.633809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aae7da8b22b4804865d92e0b1d16292'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:08.642292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aae7da8b22b4804865d92e0b1d16292'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:08.642878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aae7da8b22b4804865d92e0b1d16292'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:08.643584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aae7da8b22b4804865d92e0b1d16292'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:08.657661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aae7da8b22b4804865d92e0b1d16292'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:08.672498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1012/2034] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:08 2022] DELETE /identity/v3/users/2aae7da8b22b4804865d92e0b1d16292 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:08.677328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.677699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.677973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.699199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:08.700485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:08.762162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1023/2035] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:08.771624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:08.773161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:08.831322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab7b00-c01c-40ac-8a37-49daaea3d91c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:08.834986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:08.835613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:08.835897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:08.836181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:08.846397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:08.847829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325a2ab-a729-4b00-a065-f4666f96d697 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:08.857234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1013/2036] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:09.133936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.135473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:09.157694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:09.163007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb41e69-2bd2-4a18-831b-344f42f9ab94 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.170504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.171339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.171944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.172377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] PATH_INFO: `/v3/projects/ef59358a76914143ab89248f8297320e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.173668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef59358a76914143ab89248f8297320e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.176171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.226411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef59358a76914143ab89248f8297320e', 'request_id': 'req-50e711f8-a4f8-4088-9105-a967a4df9c1d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:09.234556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef59358a76914143ab89248f8297320e', 'request_id': 'req-50e711f8-a4f8-4088-9105-a967a4df9c1d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:09.240040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef59358a76914143ab89248f8297320e', 'request_id': 'req-50e711f8-a4f8-4088-9105-a967a4df9c1d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:09.251788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] The token cache is being invalidate because project ef59358a76914143ab89248f8297320e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:09.252073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:09.252729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-50e711f8-a4f8-4088-9105-a967a4df9c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:09.254261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1024/2037] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:09 2022] DELETE /identity/v3/projects/ef59358a76914143ab89248f8297320e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:09.259112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.259406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.259684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.281687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:09.282970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:09.343079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1014/2038] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:09.350940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.351231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.351512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.371788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:09.373057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:09.380917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1025/2039] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:09.385294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.395890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbe74f-b3f8-4605-ae0e-c547f10fcf15 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.400363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.400990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.401423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.401938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.402691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:09.402992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.404654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.416192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1015/2040] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:09.421352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.430850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1b137-9a8a-4b4a-85fc-49c1033d0626 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.436294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.436931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.437214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.437516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.438173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.440086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.467195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1026/2041] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:09.471459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.476692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:09.483245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca151707-8bed-4b39-8090-f2b909fa3482 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.542423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.543077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.543681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.543681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.544353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.547397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.575378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1016/2042] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:09.580307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.593404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb5ae8-1513-461a-aaf6-4324823d9ff1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.598106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.598717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.599001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.599277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.599992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.601621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.612042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1027/2043] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:09.616350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.627474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4616e-dbd2-4f11-b92b-ec52adf713ff admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.632059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.632697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.632982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.633257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.633942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.635775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12abbea-26c5-430f-9688-651940abad0a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.646806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1017/2044] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:09.651010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.661631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54168ac-76ba-43e4-ae7f-c8e23a75e5bd admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.666631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.667851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.668169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.668574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] PATH_INFO: `/v3/projects/f6b2a8287fe24b148ec54cb4a5af227e/users/118b8b685d524a5f8b79ffd75e148566/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.685901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6b2a8287fe24b148ec54cb4a5af227e, user_id=118b8b685d524a5f8b79ffd75e148566, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.688605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.702105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1028/2045] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:09 2022] PUT /identity/v3/projects/f6b2a8287fe24b148ec54cb4a5af227e/users/118b8b685d524a5f8b79ffd75e148566/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:09.713583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.714486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.714879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.749872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] MFA Rules not processed for user `118b8b685d524a5f8b79ffd75e148566`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:09.751127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:09.840618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1018/2046] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:09.845279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.918681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9971f83-6787-4c98-b9ce-11892a75e287 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:09.923362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:09.924012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:09.924294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:09.924602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:09.925276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:09.927443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:09.955260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1029/2047] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:09.959882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:09.972493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2a9c3-9ba2-40b3-84a9-7a1782fb423c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.029474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.030108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.030394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.030673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.031340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.033309 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.049699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:10.055881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1019/2048] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:09 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:10.060455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.070844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eecdf6-e987-4464-b867-b38aad471277 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.077516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.078179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.078463 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.078737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.079466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.081142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.083516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:10.094813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1030/2049] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:10.098977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.109093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833f688-fda5-4fbd-8e5a-b1e5da018eae admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.113706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.114319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.114604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.114881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.115598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.117252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48adafea-a81b-47a6-8a8a-324b0494ed7f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.127925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1020/2050] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:10.132456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.143719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7a7f9-3913-4eb2-bceb-801267b768a4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.148257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.148945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.149226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.149499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] PATH_INFO: `/v3/projects/19e0c623ae334173b4944e38d0603111/users/5275da71d60f42e28bb0385c634703b6/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.165668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e0c623ae334173b4944e38d0603111, user_id=5275da71d60f42e28bb0385c634703b6, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.169657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.184914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1031/2051] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:10 2022] PUT /identity/v3/projects/19e0c623ae334173b4944e38d0603111/users/5275da71d60f42e28bb0385c634703b6/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:10.195067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.196136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.196494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.228170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] MFA Rules not processed for user `5275da71d60f42e28bb0385c634703b6`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:10.230246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:10.305442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1021/2052] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:10.310671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.372561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1667bd-3054-49f8-b8a5-7876f4e884cf admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.377248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.377863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.378141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.378409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.379059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.380980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.407424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1032/2053] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:10 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:10.411545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.423021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd127f-b1eb-489d-99b7-cdc86d60ea47 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.480495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.481121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.481401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.481672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.482324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.484226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.506205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1022/2054] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:10 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:10.509994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.520770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6c334-6fe9-4708-bec1-8d52bf76343d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.525459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.526102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.526382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.526656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.527347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.528980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.540210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1033/2055] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:10.545452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.556552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5d453-bc0f-4818-82c6-94f0a7de9dcb admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.563947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.564662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.564949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.565246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] PATH_INFO: `/v3/projects/d148b10f816942ccb4536304e1acc817/users/abb66cef49f941e18826ed6769cc7f2f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.585419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d148b10f816942ccb4536304e1acc817, user_id=abb66cef49f941e18826ed6769cc7f2f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.588108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.610111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1023/2056] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:10 2022] PUT /identity/v3/projects/d148b10f816942ccb4536304e1acc817/users/abb66cef49f941e18826ed6769cc7f2f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:10.614405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.625743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67d525-5178-45a2-9af4-d151a0c809c4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.683405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.684041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.684318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.684625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.685313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.686912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7d047-8dfa-4545-9d85-7621c769109a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.697070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1034/2057] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:10.701847 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.712163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3404bca-65ee-4c99-bca3-f53f4ae04938 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.717823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.718494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.718776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.719049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] PATH_INFO: `/v3/projects/d148b10f816942ccb4536304e1acc817/users/abb66cef49f941e18826ed6769cc7f2f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.720991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d148b10f816942ccb4536304e1acc817, user_id=abb66cef49f941e18826ed6769cc7f2f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:10.723633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:10.735441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1024/2058] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:10 2022] PUT /identity/v3/projects/d148b10f816942ccb4536304e1acc817/users/abb66cef49f941e18826ed6769cc7f2f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:10.747058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.747382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.747608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:10.775888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] MFA Rules not processed for user `abb66cef49f941e18826ed6769cc7f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:10.777801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:10.816060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:10.866922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1035/2059] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:10.874787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:10.876887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:10.924946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:10.952332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f18bd-52ac-43bd-9f52-33bfde88e380 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:10.956062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:10.956707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:10.956995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:10.957273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:11.023366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:11.024869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:11.035617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1025/2060] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:14.661756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:14.672169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b3d48-8679-48d0-b6b1-e4d9a08f4059 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:14.675942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:14.676584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:14.676866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:14.677159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:14.733492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:14.734908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:14.743582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1036/2061] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:15.383316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:15.391851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6f0b7-e640-4533-bb7c-051fb30eadac service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:15.395420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:15.396024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:15.396306 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:15.396611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:15.407006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:15.408478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c49658-6bbe-4de2-8de6-00567ee0b00f service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:15.418781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1026/2062] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:16.039835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:16.041336 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:16.108000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6a04f-e2e5-4032-a8d3-3ec59b56d208 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:16.117780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:16.118349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.118636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.118913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] PATH_INFO: `/v3/users/c175b23fd9b04701b62bd7a49f7e2ee6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:16.120238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c175b23fd9b04701b62bd7a49f7e2ee6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:16.122718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:16.185539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c175b23fd9b04701b62bd7a49f7e2ee6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.186218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c175b23fd9b04701b62bd7a49f7e2ee6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.193912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c175b23fd9b04701b62bd7a49f7e2ee6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.201239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c175b23fd9b04701b62bd7a49f7e2ee6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.201904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c175b23fd9b04701b62bd7a49f7e2ee6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.207610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:16.222031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1037/2063] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:16 2022] DELETE /identity/v3/users/c175b23fd9b04701b62bd7a49f7e2ee6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:16.226429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.226717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.226996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:16.248169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:16.249774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:16.311401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1027/2064] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:16.321386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:16.322765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:16.380849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de346532-7cdc-46e8-ae82-21c2c1c14c16 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:16.384417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:16.385029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.385306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.385582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:16.394796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:16.396211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a36acf-ad31-4fe0-b822-32951cbb00a4 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:16.405522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1038/2065] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:16.693002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:16.694900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:16.718955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678213cc-5c17-419d-a47b-5618ba52795e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:16.724908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:16.736113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.736569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.736853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] PATH_INFO: `/v3/projects/3b320ff74c544841a3917d8a3628c225` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:16.738553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b320ff74c544841a3917d8a3628c225)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:16.740548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:16.835265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b320ff74c544841a3917d8a3628c225', 'request_id': 'req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.853035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b320ff74c544841a3917d8a3628c225', 'request_id': 'req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.866013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b320ff74c544841a3917d8a3628c225', 'request_id': 'req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.868609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:16.884021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] The token cache is being invalidate because project 3b320ff74c544841a3917d8a3628c225 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:16.884451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.885485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f13d1d01-d7ad-4dc5-bd0b-1edd9f3a23ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:16.888488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1028/2066] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:16 2022] DELETE /identity/v3/projects/3b320ff74c544841a3917d8a3628c225 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:16.893772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.894083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.894335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:16.921178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:16.922477 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:16.986468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1039/2067] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:16.995367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:16.995780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:16.996093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.016871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:17.018097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:17.025930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1029/2068] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:17.030954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.040462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94ccc91-3756-4b83-9885-e82d0b991557 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.045447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.046044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.046319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.046587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.047266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:17.047563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.049175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.060885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1040/2069] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:17.066353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.076300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae85b92-e03e-4046-9a96-b990c86b9745 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.081001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.081630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.081915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.082196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.082864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.084818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.116846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1030/2070] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:17 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:17.118342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.129486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04eb8e8-a589-475d-ad82-1b309170a1ba admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.188550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.189171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.189453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.189731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.190392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.192369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.215435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1041/2071] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:17 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:17.219202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.229489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe142416-0f57-40ce-b7f3-6fc1fec58bd7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.234493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.235095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.235381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.235653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.236332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.237992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.248289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1031/2072] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:17.252319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.262818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c968-41b5-42a0-b9f5-0d12874cb035 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.268443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.269081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.269365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.269636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.270364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.271965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3534981b-e0cf-438e-8962-f2e0d81abf07 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.274819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:17.289908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1042/2073] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:17.295624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.308382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecb03e-fd70-4eec-810b-8849cb441f2d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.316693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.317388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.317674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.317955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] PATH_INFO: `/v3/projects/06b449da3f4248579c6a12f07c7f5b55/users/c10b35e8364c4789b7a926888f1e7234/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.338613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b449da3f4248579c6a12f07c7f5b55, user_id=c10b35e8364c4789b7a926888f1e7234, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.341733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.357146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1032/2074] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:17 2022] PUT /identity/v3/projects/06b449da3f4248579c6a12f07c7f5b55/users/c10b35e8364c4789b7a926888f1e7234/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:17.362472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.362767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.363045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.383024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:17.384244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:17.447788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1043/2075] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:17.458771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:17.460656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:17.521256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a2e4b4-09da-4e13-9ab4-537d694e6fb3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:17.524873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:17.527831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:17.527831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:17.527831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:17.537538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:17.539097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa24e58-1874-4716-b107-0c4a2c463679 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:17.550182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1033/2076] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:20.724451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:20.726073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:20.751836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d04767-0dde-418e-930b-05ed08d7f967 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:20.765417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:20.766089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:20.766374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:20.766651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] PATH_INFO: `/v3/users/118b8b685d524a5f8b79ffd75e148566` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:20.770680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] RBAC: Authorizing `identity:delete_user(user_id=118b8b685d524a5f8b79ffd75e148566)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:20.773673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:20.862727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118b8b685d524a5f8b79ffd75e148566'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:20.863863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118b8b685d524a5f8b79ffd75e148566'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:20.886461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118b8b685d524a5f8b79ffd75e148566'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:20.904663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118b8b685d524a5f8b79ffd75e148566'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:20.905728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118b8b685d524a5f8b79ffd75e148566'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:20.938322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1044/2077] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:20 2022] DELETE /identity/v3/users/118b8b685d524a5f8b79ffd75e148566 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:20.943266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:20.944437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:20.944943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:20.973467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:20.975033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:21.055480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1034/2078] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:21.069409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:21.072268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:21.152948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4e938-6dc5-41bc-98e4-0592916238df admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:21.157669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:21.158481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:21.158820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:21.159088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:21.173280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:21.175097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:21.190959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1045/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:21.989316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:21.991424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:22.016040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918c359-00f5-45be-a5e1-957f0473d1cb None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:22.028356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:22.029652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:22.030147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:22.030647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] PATH_INFO: `/v3/projects/f6b2a8287fe24b148ec54cb4a5af227e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:22.032327 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6b2a8287fe24b148ec54cb4a5af227e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:22.034685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:22.108784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:22.136654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6b2a8287fe24b148ec54cb4a5af227e', 'request_id': 'req-39b7dad1-5552-447f-b48c-f4f8bcd27020'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.147822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6b2a8287fe24b148ec54cb4a5af227e', 'request_id': 'req-39b7dad1-5552-447f-b48c-f4f8bcd27020'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.160042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6b2a8287fe24b148ec54cb4a5af227e', 'request_id': 'req-39b7dad1-5552-447f-b48c-f4f8bcd27020'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.181798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] The token cache is being invalidate because project f6b2a8287fe24b148ec54cb4a5af227e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:22.183591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.184747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-39b7dad1-5552-447f-b48c-f4f8bcd27020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.189975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1035/2080] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:21 2022] DELETE /identity/v3/projects/f6b2a8287fe24b148ec54cb4a5af227e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:22.194010 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:22.208200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584a4c-9420-476d-9705-646baee30e7b service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:22.221002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:22.286358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:22.287378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:22.287888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:22.288658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] PATH_INFO: `/v3/users/5275da71d60f42e28bb0385c634703b6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:22.290507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5275da71d60f42e28bb0385c634703b6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:22.293589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:22.396394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5275da71d60f42e28bb0385c634703b6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.398812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5275da71d60f42e28bb0385c634703b6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.414436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5275da71d60f42e28bb0385c634703b6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.437141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5275da71d60f42e28bb0385c634703b6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.438696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5275da71d60f42e28bb0385c634703b6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:22.467756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1046/2081] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:22 2022] DELETE /identity/v3/users/5275da71d60f42e28bb0385c634703b6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:22.761547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:22.761918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:22.762101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:22.796870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] MFA Rules not processed for user `c10b35e8364c4789b7a926888f1e7234`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:22.798549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:22.885193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1036/2082] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:22.890789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:22.891804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:22.892795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:22.893801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:22.992028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d590ed-c29c-4910-83e8-818c0bd58c02 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:22.999613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:22.999829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78d81dd-6166-4819-a0b3-87eff6602869 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.000367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.000646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.001079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.001938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.004038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.005107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.005854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.006236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.006614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] PATH_INFO: `/v3/projects/19e0c623ae334173b4944e38d0603111` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.007911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e0c623ae334173b4944e38d0603111)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.009896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.032668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1037/2083] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:22 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:23.036808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.047626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:23.064929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e0c623ae334173b4944e38d0603111', 'request_id': 'req-b11587b7-a8dd-4085-86b1-0706f522a5e7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.072547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e0c623ae334173b4944e38d0603111', 'request_id': 'req-b11587b7-a8dd-4085-86b1-0706f522a5e7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.079086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:23.080775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e0c623ae334173b4944e38d0603111', 'request_id': 'req-b11587b7-a8dd-4085-86b1-0706f522a5e7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.095775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] The token cache is being invalidate because project 19e0c623ae334173b4944e38d0603111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:23.096198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.096959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.098534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1047/2084] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:22 2022] DELETE /identity/v3/projects/19e0c623ae334173b4944e38d0603111 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:23.102357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.103769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:23.111311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876175f-9300-4d36-82ba-2a22bd10da94 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.168197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.168471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11587b7-a8dd-4085-86b1-0706f522a5e7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.168842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.169238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.169519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.170177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.172113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.173150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.173777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.174059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.174327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] PATH_INFO: `/v3/users/abb66cef49f941e18826ed6769cc7f2f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.175722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb66cef49f941e18826ed6769cc7f2f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.177659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.198045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1038/2085] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:23 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:23.202534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.204426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:23.217721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700d2b2-0294-41b5-9b62-4306ef8ba5e3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.222442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.223096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.223381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.223661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.224366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.226043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.230418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb66cef49f941e18826ed6769cc7f2f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.231294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb66cef49f941e18826ed6769cc7f2f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.236477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1039/2086] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:23.239413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb66cef49f941e18826ed6769cc7f2f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.240771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.243058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:23.247170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb66cef49f941e18826ed6769cc7f2f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.247977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb66cef49f941e18826ed6769cc7f2f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.255988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4e9a2-c6e7-4042-bdc0-65b5d7391381 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.260622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.261869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.262150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.262421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] PATH_INFO: `/v3/projects/762735c015754691b9dd41f33bde68c0/users/5322d6d8a0d54487bb7f3ff30f4c7f8f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.266880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1048/2087] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:23 2022] DELETE /identity/v3/users/abb66cef49f941e18826ed6769cc7f2f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:23.285886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] RBAC: Authorizing `identity:create_grant(project_id=762735c015754691b9dd41f33bde68c0, user_id=5322d6d8a0d54487bb7f3ff30f4c7f8f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.288971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.304514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1040/2088] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:23 2022] PUT /identity/v3/projects/762735c015754691b9dd41f33bde68c0/users/5322d6d8a0d54487bb7f3ff30f4c7f8f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:23.310367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.360362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:23.385876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990648-1ed1-43d2-8350-5f3dc1a9f902 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.475068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.476509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.476812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.477424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.478152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.480546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.503001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1049/2089] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:23.507496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.522441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c6471-ca2b-4df2-a977-8ca046d9188e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.538645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.540613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.543210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.543882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] PATH_INFO: `/v3/projects/762735c015754691b9dd41f33bde68c0/users/5322d6d8a0d54487bb7f3ff30f4c7f8f/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.551419 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=762735c015754691b9dd41f33bde68c0, user_id=5322d6d8a0d54487bb7f3ff30f4c7f8f, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.571873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee133ee-47e7-485c-8e1a-6edec2b9f1ad admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.601694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1041/2090] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:23 2022] PUT /identity/v3/projects/762735c015754691b9dd41f33bde68c0/users/5322d6d8a0d54487bb7f3ff30f4c7f8f/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:23.723998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:23.726683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:23.750052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7acf7f7-82db-4a41-94ad-25f2f71ef9ee admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:23.872041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:23.872829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:23.873120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:23.873398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] PATH_INFO: `/v3/projects/d148b10f816942ccb4536304e1acc817` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:23.874980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d148b10f816942ccb4536304e1acc817)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:23.877867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:23.954349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd148b10f816942ccb4536304e1acc817', 'request_id': 'req-310d5906-a551-4d50-a139-c8287e111d8d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.973445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd148b10f816942ccb4536304e1acc817', 'request_id': 'req-310d5906-a551-4d50-a139-c8287e111d8d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:23.984021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd148b10f816942ccb4536304e1acc817', 'request_id': 'req-310d5906-a551-4d50-a139-c8287e111d8d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:24.011929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] The token cache is being invalidate because project d148b10f816942ccb4536304e1acc817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:24.012462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:24.013537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-310d5906-a551-4d50-a139-c8287e111d8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:24.021725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1050/2091] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:23 2022] DELETE /identity/v3/projects/d148b10f816942ccb4536304e1acc817 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:24.022195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.022634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.022753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.052239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:24.053547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:24.141324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1042/2092] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:24.151295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.151295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.152080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.187832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:24.190650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:24.213589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1051/2093] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:24.218232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.229671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c86e6d-547b-49c7-b5a1-79b7ff0cce7e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.235732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.236437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.236933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.237225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.239493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:24.239493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.240828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.260475 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1043/2094] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:24.264734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.277972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1a97d3-0f54-451e-ad08-9a70a26aa149 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.285505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.286410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.287268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.287268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.287862 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.290894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.300191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:24.340410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1052/2095] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:24 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:24.342077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.353078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736840c9-17ab-46ab-a221-8c1bbf42431c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.359569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:24.444186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.444850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.445183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.445464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.446150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.448643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.483744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.485035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1044/2096] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:24 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:24.502249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33552736-8c34-4cd9-9849-f857f69831f8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.510142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.511725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.512074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.512478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.518058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.518058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.528046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1053/2097] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:24.531920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.545293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e98028b-567b-4489-bb22-38237f0beba6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.551595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.552524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.552946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.553404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.554405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.556661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63afaa22-b119-44d9-9ee4-608050b802ee admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.570981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1045/2098] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:24.582136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.609507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da22c4-0d51-4ad8-86c9-6f9bba55852b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:24.625416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:24.626146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.626612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.626902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] PATH_INFO: `/v3/projects/71a188048d0b4bc3a68032746259e533/users/a871356deb224b9cb09a28848762cee0/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.664273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a188048d0b4bc3a68032746259e533, user_id=a871356deb224b9cb09a28848762cee0, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:24.667395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:24.710906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1054/2099] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:24 2022] PUT /identity/v3/projects/71a188048d0b4bc3a68032746259e533/users/a871356deb224b9cb09a28848762cee0/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:24.712722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea54cba-2d62-4c7b-bba7-c080a47bc18c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:24.713362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea54cba-2d62-4c7b-bba7-c080a47bc18c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:24.714872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea54cba-2d62-4c7b-bba7-c080a47bc18c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:24.775689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9ea54cba-2d62-4c7b-bba7-c080a47bc18c None None] MFA Rules not processed for user `a871356deb224b9cb09a28848762cee0`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:24.777514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9ea54cba-2d62-4c7b-bba7-c080a47bc18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:24.893341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1046/2100] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:24.906756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:24.908565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:25.008682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1502b5a-a6fa-4f5d-bad4-23a9c5eb5acf admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:25.013932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:25.014908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:25.015423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:25.015902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:25.029745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:25.032181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb62853-981b-4244-bb5d-cb3d702f07d6 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:25.047733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1055/2101] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:29.167659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:29.172049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:29.173491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:29.176680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1047/2102] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:56:29 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:56:29.210341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:29.210883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:29.210883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:29.239905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] MFA Rules not processed for user `72012ef48a864233970d32552c403409`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:29.240252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:29.276081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:29.279256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:29.399025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1056/2103] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:56:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:29.404626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39544bcd-0fa4-467c-ae0b-7bc1c1dc0229 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:29.412770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:29.414182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:29.414625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:29.415155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] PATH_INFO: `/v3/users/cfdbada2f1a74976b42d299a813335cf` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:29.416359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:29.417883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfdbada2f1a74976b42d299a813335cf)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:29.418205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:29.420456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:29.438724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5be1f0-8ab6-4905-88f4-5d2a5003bca7 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:29.451955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:29.452955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:29.453239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:29.453597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:29.466302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:29.468236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:29.472633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:29.491691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1057/2104] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:29.507228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfdbada2f1a74976b42d299a813335cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:29.531956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfdbada2f1a74976b42d299a813335cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:29.533461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfdbada2f1a74976b42d299a813335cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:29.535824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfdbada2f1a74976b42d299a813335cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:29.569890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-50001995-5857-436a-a820-c8ad7d28e297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfdbada2f1a74976b42d299a813335cf'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:29.588368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1048/2105] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:29 2022] DELETE /identity/v3/users/cfdbada2f1a74976b42d299a813335cf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:29.858122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:29.883387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72693e-6c59-4ef8-91cd-268069ef6a3e service placement] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:29.951928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:29.999037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.005319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.005678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.006028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] PATH_INFO: `/v3/projects/281963666cbf4245a97cbb213a3fa0e8` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.008770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281963666cbf4245a97cbb213a3fa0e8)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.011735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.097629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281963666cbf4245a97cbb213a3fa0e8', 'request_id': 'req-add3d3e3-81ad-4b57-ac98-0df4d0846160'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:30.107983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281963666cbf4245a97cbb213a3fa0e8', 'request_id': 'req-add3d3e3-81ad-4b57-ac98-0df4d0846160'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:30.116909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281963666cbf4245a97cbb213a3fa0e8', 'request_id': 'req-add3d3e3-81ad-4b57-ac98-0df4d0846160'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:30.131699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] The token cache is being invalidate because project 281963666cbf4245a97cbb213a3fa0e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:30.132973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:30.133866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-add3d3e3-81ad-4b57-ac98-0df4d0846160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:30.140784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1058/2106] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:29 2022] DELETE /identity/v3/projects/281963666cbf4245a97cbb213a3fa0e8 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:30.142550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.142988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.143141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.165504 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:30.166795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:30.241931 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1049/2107] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:30.250129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.250644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.250749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.273025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:30.274384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:30.284821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1059/2108] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:30.289215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.301838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c30e256-8fdd-4496-806f-d4e7c2fd23ec None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.308882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.310037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.310363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.310714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.312219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:30.312219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.316413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.332237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1050/2109] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:56:30.333388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.342944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4b230-c5fa-4df3-8cc3-7fac03dd0431 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.351276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.351915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.352202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.352520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.353400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.356163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.405579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1060/2110] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:30.406692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.421370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85843193-1ca4-44cd-9b87-368cc60555cb admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.497242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.497474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.504481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.505234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.506186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.510051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.560009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1051/2111] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:30.566444 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.581542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b94d8-d69e-4f18-a3e5-4dfb46bd1fc7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.591166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.591854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.593704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.594626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.596038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.597899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e8508-aaaf-46a4-86a4-954000b7900e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.613452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1061/2112] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:30.617941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.633174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4818ebb-94a8-4cee-8bc7-10adcc9d9594 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.649250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.649928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.650215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.650494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.651226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.653759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.654044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.654151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.654318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.677958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1052/2113] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:30.682077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.698181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:30.718395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c11fed-4375-4d34-ae6b-93388199ec9a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:30.720943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] MFA Rules not processed for user `5322d6d8a0d54487bb7f3ff30f4c7f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:30.722297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:30.726057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:30.727274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.728362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.728998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] PATH_INFO: `/v3/projects/06ed07be36cd448fa85d19e6e4b6e1f4/users/1c5045fab25447b5ab110941fd50144a/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.754771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ed07be36cd448fa85d19e6e4b6e1f4, user_id=1c5045fab25447b5ab110941fd50144a, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:30.758189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce8b4-3b56-4f1e-a896-5eaad0320591 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:30.796556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1053/2114] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:30 2022] PUT /identity/v3/projects/06ed07be36cd448fa85d19e6e4b6e1f4/users/1c5045fab25447b5ab110941fd50144a/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:30.811015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:30.812830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:30.814804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:30.842140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1062/2115] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:30.860534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:30.860534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:30.878836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:30.891503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] MFA Rules not processed for user `1c5045fab25447b5ab110941fd50144a`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:30.892813 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:30.937768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147fb37-d362-4fdf-ba93-a0c9086f13eb None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.011927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.012761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.013045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.013318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.015236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:31.017272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1054/2116] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:31.021491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.120267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.122609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.134161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1063/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:31.140008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53102f54-c965-48e8-a3f8-7bebc810890c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.145051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.145680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.145964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.146242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.146910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.148864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00304f4b-8b53-48b3-b94a-20c4c34c8d54 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.188920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1055/2118] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:31 2022] POST /identity/v3/projects => generated 369 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:31.189780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.209757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b55d-a165-405c-b055-81538442bf5a service glance] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.291569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.292496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.292985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.303530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.304229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.304674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.304976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.305668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.307755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.335085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:31.336119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:31.387747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1064/2119] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:31 2022] POST /identity/v3/users => generated 291 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:31.389593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.405786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dc18-d61b-4fe2-b077-a9900b4996a0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.411827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.412702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.412986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.413266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.414556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.416705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.416939 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:31.425837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:31.431439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1056/2120] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:56:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:31.441747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.443845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1065/2121] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:31.461604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3102b1-93f9-4b8c-be6a-0d17f669f43c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.467560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.468249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.468562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.469003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] PATH_INFO: `/v3/projects/ed522a0dcabd48e99329622bca76c698/users/1a83353e349e41029552fb23141de682/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.489972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.492101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:31.492660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed522a0dcabd48e99329622bca76c698, user_id=1a83353e349e41029552fb23141de682, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.499187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.535800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1057/2122] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:31 2022] PUT /identity/v3/projects/ed522a0dcabd48e99329622bca76c698/users/1a83353e349e41029552fb23141de682/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:31.539384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.607328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b87ddc3-3d3f-4e21-ab19-394e458de494 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.632810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70ba4e4-fe4a-45d3-ad00-2a2e99f2eea6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.638720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.639023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.639238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.639663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.640408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.642046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.653304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1058/2123] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:31.658505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.670196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a96c42-8e0b-472f-b66a-6606f03e90ce admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:31.675810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.677109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.677491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.678082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] PATH_INFO: `/v3/projects/ed522a0dcabd48e99329622bca76c698/users/1a83353e349e41029552fb23141de682/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.680383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed522a0dcabd48e99329622bca76c698, user_id=1a83353e349e41029552fb23141de682, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.683907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c55b2ac-f903-453d-a7f8-6a1d31716d39 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.704598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1059/2124] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:31 2022] PUT /identity/v3/projects/ed522a0dcabd48e99329622bca76c698/users/1a83353e349e41029552fb23141de682/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:31.709142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:31.709790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.710072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.710775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.714137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:31.714426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:31.714705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:31.752113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] MFA Rules not processed for user `1a83353e349e41029552fb23141de682`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:31.753549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:31.825198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:31.826874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:31.843139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1066/2125] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:56:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:56:31.875736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1060/2126] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:31.887902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:31.892937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:32.095000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:32.107790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caab3b-4ffc-4538-919b-b1546ad30b76 service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:32.196496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:32.197139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:32.197426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:32.197704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:32.217294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:32.218998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b918e36-f66b-4e71-a525-d14c79162cde service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:32.232533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1067/2127] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:34.315822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:34.377468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:34.411096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f765aa-d7b5-45cb-bcee-38b0f2d7c460 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:34.418779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:34.419505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:34.419787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:34.420067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] PATH_INFO: `/v3/users/1c5045fab25447b5ab110941fd50144a` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:34.423528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5045fab25447b5ab110941fd50144a)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:34.425771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:34.520774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5045fab25447b5ab110941fd50144a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:34.545787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5045fab25447b5ab110941fd50144a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:34.547914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5045fab25447b5ab110941fd50144a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:34.549441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5045fab25447b5ab110941fd50144a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:34.575977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5045fab25447b5ab110941fd50144a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:34.599941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1061/2128] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:34 2022] DELETE /identity/v3/users/1c5045fab25447b5ab110941fd50144a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:34.601939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:34.602434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:34.602515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:34.629833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:34.631063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:34.732576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1068/2129] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:34.744489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:34.746823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:34.827619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc4e13-dfff-4d08-8af2-ce73292e91e8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:34.831618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:34.832383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:34.832744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:34.833033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:34.843469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:34.845160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:34.856247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1062/2130] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:35.365851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:35.367495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:35.399960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9068c-070a-4f47-8744-a365ac105c84 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:35.410676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:35.411725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:35.412212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:35.412762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] PATH_INFO: `/v3/projects/06ed07be36cd448fa85d19e6e4b6e1f4` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:35.414691 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ed07be36cd448fa85d19e6e4b6e1f4)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:35.422454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:35.547003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ed07be36cd448fa85d19e6e4b6e1f4', 'request_id': 'req-5f88bdf0-149b-40f7-9190-eaffadb8df72'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.565127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ed07be36cd448fa85d19e6e4b6e1f4', 'request_id': 'req-5f88bdf0-149b-40f7-9190-eaffadb8df72'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.574763 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ed07be36cd448fa85d19e6e4b6e1f4', 'request_id': 'req-5f88bdf0-149b-40f7-9190-eaffadb8df72'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.602007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] The token cache is being invalidate because project 06ed07be36cd448fa85d19e6e4b6e1f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:35.602623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.603013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.608544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1069/2131] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:35 2022] DELETE /identity/v3/projects/06ed07be36cd448fa85d19e6e4b6e1f4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:35.611191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:35.629016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cb515-017a-4175-bed8-1846c21ff28d service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:35.705629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:35.707253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:35.707548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:35.707828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] PATH_INFO: `/v3/users/1a83353e349e41029552fb23141de682` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:35.717041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a83353e349e41029552fb23141de682)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:35.719733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:35.725348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:35.811006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a83353e349e41029552fb23141de682'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.840030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a83353e349e41029552fb23141de682'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.840926 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a83353e349e41029552fb23141de682'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.844424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a83353e349e41029552fb23141de682'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.874267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d28b3400-61e8-44e2-bcd6-3407997090e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a83353e349e41029552fb23141de682'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:35.911320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1063/2132] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:35 2022] DELETE /identity/v3/users/1a83353e349e41029552fb23141de682 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:36.412476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:36.414369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:36.458111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88bdf0-149b-40f7-9190-eaffadb8df72 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:36.530710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:36.579363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:36.580207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:36.580804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:36.581176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] PATH_INFO: `/v3/projects/ed522a0dcabd48e99329622bca76c698` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:36.583761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed522a0dcabd48e99329622bca76c698)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:36.586928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:36.665165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed522a0dcabd48e99329622bca76c698', 'request_id': 'req-8411924e-b582-4299-838e-c0019871ee96'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:36.677932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed522a0dcabd48e99329622bca76c698', 'request_id': 'req-8411924e-b582-4299-838e-c0019871ee96'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:36.685191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed522a0dcabd48e99329622bca76c698', 'request_id': 'req-8411924e-b582-4299-838e-c0019871ee96'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:36.698595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] The token cache is being invalidate because project ed522a0dcabd48e99329622bca76c698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:36.698970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:36.699558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8411924e-b582-4299-838e-c0019871ee96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:36.701135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1070/2133] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:36 2022] DELETE /identity/v3/projects/ed522a0dcabd48e99329622bca76c698 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:36.708383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:36.708948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:36.709157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:36.733952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:36.735486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:36.826852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1064/2134] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:36.836254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:36.837369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:36.837655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:36.863888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:36.865680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:36.889073 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1071/2135] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:36.889658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:36.905022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a11055-c14e-4c67-b395-fb4ff4303941 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:36.919454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:36.920108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:36.920427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:36.920704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:36.921406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:36.921706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:36.923331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:36.959003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1065/2136] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:36.962158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:36.973449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252d003-5765-4296-bfe6-8de2bf2148fe None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:36.981890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:36.982537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:36.982819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:36.983099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:36.983764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:36.985768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.032381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1072/2137] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:36 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:37.035620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.049049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61537024-e49a-40f7-9b78-bc779194090e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.123998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.124733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.125097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.125448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.126196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.128430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.155251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:37.163838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1066/2138] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:37 2022] POST /identity/v3/users => generated 284 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:37.168901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.184094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed20421-c37b-4ecc-9557-7a80bb02ad4f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.189605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.190427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.190679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.190960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.191655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.193445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.209989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1073/2139] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:37.214572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.227452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592448e1-dd78-42a3-8daa-2ae78b40dda3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.234941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.235518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.235801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.236074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.236821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.239242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06277d15-c593-4d7c-9f10-17a26cc61204 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.258731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.272396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1067/2140] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:37.273847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142be8a-fc25-44fd-bc82-57bf3c17c96f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.281322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.282147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.282448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.282731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] PATH_INFO: `/v3/projects/e6027678f1174c73a9e83c21a61b854e/users/ff14386b8b6d451a9467d3dadaf78a02/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.303574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6027678f1174c73a9e83c21a61b854e, user_id=ff14386b8b6d451a9467d3dadaf78a02, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.306357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.334050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1074/2141] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:37 2022] PUT /identity/v3/projects/e6027678f1174c73a9e83c21a61b854e/users/ff14386b8b6d451a9467d3dadaf78a02/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:37.341466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.341758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.342042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.384768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] MFA Rules not processed for user `ff14386b8b6d451a9467d3dadaf78a02`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:37.389590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:37.490598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1068/2142] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:37.495796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.626649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:37.635503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3fe4b-b881-468b-8519-6266a3e8e06b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.644325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.645671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.646054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.646423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.647190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.649436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.705417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1075/2143] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:37 2022] POST /identity/v3/projects => generated 367 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:37.709378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.725391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c325-5703-4fff-a7f9-17d9a23fd251 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.814937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.816092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.816709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.817564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.818431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.820628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.883400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1069/2144] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:37 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:37.884685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.905900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841e5ac-ac0a-4ef8-a8d3-2e9e7f1bc2c8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.918893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.919520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.919803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.920080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:37.921246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:37.923269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:37.957305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1076/2145] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:37.961489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:37.982891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a109150-87b6-49fa-816f-eab04d033312 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:37.988290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:37.989335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:37.989661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:37.990248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] PATH_INFO: `/v3/projects/8dec22bab9ab45cfb64f4b75097442e2/users/796020ad3bea4bc3a9b72844526617d7/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:38.052583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dec22bab9ab45cfb64f4b75097442e2, user_id=796020ad3bea4bc3a9b72844526617d7, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:38.062571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:38.117892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1070/2146] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:37 2022] PUT /identity/v3/projects/8dec22bab9ab45cfb64f4b75097442e2/users/796020ad3bea4bc3a9b72844526617d7/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:38.122521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:38.135048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bb46f-ac43-4ac4-837c-fe83543c2f92 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:38.278283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:38.290612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:38.291119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:38.291588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:38.292631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:38.313548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9aa3ac-eb32-45bb-a85c-058564422633 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:38.336056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1077/2147] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:38.344416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:38.361248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc0cc6c-6eaa-47a5-809f-657648219954 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:38.406290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:38.407088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:38.407536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:38.408100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] PATH_INFO: `/v3/projects/8dec22bab9ab45cfb64f4b75097442e2/users/796020ad3bea4bc3a9b72844526617d7/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:38.410531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dec22bab9ab45cfb64f4b75097442e2, user_id=796020ad3bea4bc3a9b72844526617d7, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:38.413508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:38.449112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1071/2148] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:38 2022] PUT /identity/v3/projects/8dec22bab9ab45cfb64f4b75097442e2/users/796020ad3bea4bc3a9b72844526617d7/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:38.450285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:38.450728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:38.451016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:38.514935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] MFA Rules not processed for user `796020ad3bea4bc3a9b72844526617d7`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:38.517538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:38.621190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1078/2149] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:38.633438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:38.637041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:38.727491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:38.793598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438e4a4-6afb-4feb-ae0b-20f5f4c4850b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:38.798820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:38.799452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:38.799732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:38.800005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:38.811121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:38.812603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c11eb-1f2a-405d-97db-159bb828fa3f service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:38.822527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1072/2150] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:38.937066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:38.954237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeaa8c2-f3de-4209-824f-2efdc36d6e1f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:38.959270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:38.968848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:38.969357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:38.969943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:39.066144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:39.097028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:39.100895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88d84-66ec-4a6a-afe6-284608d25bb6 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:39.121537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1079/2151] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:43.634899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:43.635501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:43.635600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:43.655216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] MFA Rules not processed for user `c10b35e8364c4789b7a926888f1e7234`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:43.656521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:43.713928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1073/2152] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:45.913083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:45.913609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:45.913710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:45.936757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:45.937982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:45.999962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1080/2153] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:56:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:46.012145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:46.013617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:46.111453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8011d-7494-4f72-ac97-da1010ccd097 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:46.117883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:46.118518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:46.118993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:46.119276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:46.130718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:46.133245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef42efc-da17-4a7f-a589-b2ce08e39d2f service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:46.142490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1074/2154] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:56:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:56:46.950123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:47.115037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1c6cb-2226-4b36-9b31-9663629baa76 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:47.127076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:47.127856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:47.128238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:47.128642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] PATH_INFO: `/v3/users/ff14386b8b6d451a9467d3dadaf78a02` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:47.130002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff14386b8b6d451a9467d3dadaf78a02)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:47.132784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:47.214799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff14386b8b6d451a9467d3dadaf78a02'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.215629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff14386b8b6d451a9467d3dadaf78a02'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.231048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff14386b8b6d451a9467d3dadaf78a02'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.242783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff14386b8b6d451a9467d3dadaf78a02'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.243514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff14386b8b6d451a9467d3dadaf78a02'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.268093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1081/2155] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:46 2022] DELETE /identity/v3/users/ff14386b8b6d451a9467d3dadaf78a02 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:47.273005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:47.273477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:47.273589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:47.294704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:47.296225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:47.368853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1075/2156] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:47.375997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:47.377457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:47.393320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:47.455273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1395e-f05c-4770-98d9-15c14399a449 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:47.459258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:47.459923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:47.460199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:47.460521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:47.471726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:47.473753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:47.485421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1082/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:47.815957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:47.817399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:47.843036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962490-9a52-4c11-ab99-f886a9965834 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:47.849392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:47.850171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:47.850455 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:47.850733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] PATH_INFO: `/v3/projects/e6027678f1174c73a9e83c21a61b854e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:47.851924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6027678f1174c73a9e83c21a61b854e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:47.854442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:47.858583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:47.916215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6027678f1174c73a9e83c21a61b854e', 'request_id': 'req-bebe8308-42ff-4d66-a2da-5a88abfdf55c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.925227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6027678f1174c73a9e83c21a61b854e', 'request_id': 'req-bebe8308-42ff-4d66-a2da-5a88abfdf55c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.932763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6027678f1174c73a9e83c21a61b854e', 'request_id': 'req-bebe8308-42ff-4d66-a2da-5a88abfdf55c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.945069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] The token cache is being invalidate because project e6027678f1174c73a9e83c21a61b854e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:47.945584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.946172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:47.947746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1076/2158] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:47 2022] DELETE /identity/v3/projects/e6027678f1174c73a9e83c21a61b854e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:47.952024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:47.961820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac167ae1-2ecb-4577-80da-9bc4303cdb6c service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:48.035048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:48.035697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:48.035978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:48.036252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] PATH_INFO: `/v3/users/796020ad3bea4bc3a9b72844526617d7` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:48.037451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796020ad3bea4bc3a9b72844526617d7)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:48.039389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:48.096669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796020ad3bea4bc3a9b72844526617d7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.097621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796020ad3bea4bc3a9b72844526617d7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.107244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796020ad3bea4bc3a9b72844526617d7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.119890 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796020ad3bea4bc3a9b72844526617d7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.121091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16b7ef84-5e29-4b99-bc0a-09418ac8d623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796020ad3bea4bc3a9b72844526617d7'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.150480 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1083/2159] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:47 2022] DELETE /identity/v3/users/796020ad3bea4bc3a9b72844526617d7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:48.606192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:48.608924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:48.639850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe8308-42ff-4d66-a2da-5a88abfdf55c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:48.722767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:48.723339 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:48.723617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:48.723889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] PATH_INFO: `/v3/projects/8dec22bab9ab45cfb64f4b75097442e2` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:48.725660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dec22bab9ab45cfb64f4b75097442e2)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:48.727607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:48.774709 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dec22bab9ab45cfb64f4b75097442e2', 'request_id': 'req-56ff4f86-b379-42ed-8c5f-c49392ecdc03'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.783641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dec22bab9ab45cfb64f4b75097442e2', 'request_id': 'req-56ff4f86-b379-42ed-8c5f-c49392ecdc03'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.790871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dec22bab9ab45cfb64f4b75097442e2', 'request_id': 'req-56ff4f86-b379-42ed-8c5f-c49392ecdc03'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.803130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] The token cache is being invalidate because project 8dec22bab9ab45cfb64f4b75097442e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:48.803459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.804081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-56ff4f86-b379-42ed-8c5f-c49392ecdc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:48.805819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1077/2160] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:48 2022] DELETE /identity/v3/projects/8dec22bab9ab45cfb64f4b75097442e2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:48.811454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:48.811745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:48.812025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:48.834471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:48.835916 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:48.864702 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:48.899510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1084/2161] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:48.907401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:48.907780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:48.908243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:48.943106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:48.944704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:48.953506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1078/2162] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:48.957730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:48.967646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f163df-dfe7-4f96-8862-9a2dae6f6d7d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:48.972392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:48.973064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:48.973294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:48.973570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:48.974263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:48.974569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:48.976366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:48.988687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1085/2163] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 20:56:48.995891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.025310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed266a2-c8cb-43f5-988d-2e7eb8d27103 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.046143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.047125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.047398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.047701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.048463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.052009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.073273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:49.118219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1079/2164] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:48 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:49.122747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.133532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf9a66-236f-44c8-b1fa-50d33be26b22 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.201781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.202406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.202688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.202960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.203627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.205672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.229835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1086/2165] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:49 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:49.233979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.245034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558a4fbe-c43b-4ba9-b502-1a820136688d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.250516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.251152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.251439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.251717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.252441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.254066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.270003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1080/2166] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:49.274245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.286355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d49452-83b7-4641-a188-913fa85915a0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.290919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.291676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.292107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.292413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.293112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.294711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21dbbda-9bf7-4172-9176-7c73c1356536 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.305968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1087/2167] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:49.310171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.320762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc5c83-c95c-4b46-bf00-93b7f062b2b7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.326303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.327000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.327287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.327577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] PATH_INFO: `/v3/projects/551bc3a1de8d4b69946e0f93e6baf7c8/users/e3a030d8c10d4ecc966ca7145fece349/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.343728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=551bc3a1de8d4b69946e0f93e6baf7c8, user_id=e3a030d8c10d4ecc966ca7145fece349, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.346430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.359568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1081/2168] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:49 2022] PUT /identity/v3/projects/551bc3a1de8d4b69946e0f93e6baf7c8/users/e3a030d8c10d4ecc966ca7145fece349/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:49.371183 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.371471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.371746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.405946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] MFA Rules not processed for user `e3a030d8c10d4ecc966ca7145fece349`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:49.406634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.408625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:49.420117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:49.500627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1088/2169] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:49.501326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db365ef-489d-4259-9b7d-bc9ffafd13c3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.504914 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.507758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.509300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.509761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.510789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] PATH_INFO: `/v3/users/a871356deb224b9cb09a28848762cee0` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.512672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a871356deb224b9cb09a28848762cee0)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.514824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.518246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d9838-9716-4517-b506-a7677e47dde4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.533131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.533131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.533131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.533469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.534050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.536119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.608787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a871356deb224b9cb09a28848762cee0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:49.620574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1089/2170] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:49 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:56:49.621399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a871356deb224b9cb09a28848762cee0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:49.622212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.624283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:49.630522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a871356deb224b9cb09a28848762cee0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:49.632038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a871356deb224b9cb09a28848762cee0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:49.638226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:49.699902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3102d4ac-9d0f-4765-bee9-ddaa23babae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a871356deb224b9cb09a28848762cee0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:49.736047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1082/2171] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:49 2022] DELETE /identity/v3/users/a871356deb224b9cb09a28848762cee0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:49.740757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.741175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.741562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.766405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba14636-0481-4286-8348-15cd54f56a01 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.784197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:49.785646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:49.835858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:49.852053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.852884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.853382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.853744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.855494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.857586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.895363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1090/2172] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:49 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:49.895637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1083/2173] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:49.895787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.905911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.909531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:49.918170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519cfb1-619a-4bcc-b3ca-5a4d1b82ef2e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.923846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.924649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.924933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.925761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.926470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.928089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.939728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1091/2174] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:49.943712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:49.955265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dbd359-2ead-4ac2-8104-7042097a246a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.960417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.961125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.961368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.961649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] PATH_INFO: `/v3/projects/e01e1f05064247abad28bb5c5dc780e3/users/4490d9b438fc4ec5a2d96111e594b2c0/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.979191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01e1f05064247abad28bb5c5dc780e3, user_id=4490d9b438fc4ec5a2d96111e594b2c0, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.979793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1705d6-5243-442a-b4a7-cd3b560d4505 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:49.982130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:49.985033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:49.986435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:49.986785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:49.987217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:49.996655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1092/2175] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:49 2022] PUT /identity/v3/projects/e01e1f05064247abad28bb5c5dc780e3/users/4490d9b438fc4ec5a2d96111e594b2c0/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:49.998375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:49.999842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:50.000576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:50.011063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1084/2176] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:50.116063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b0e33-1927-4451-bb5d-cb8fc7bfda81 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:50.123416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:50.124196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:50.124598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:50.124978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:50.125786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:50.139127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506de790-236b-4bf3-b80b-f82e4acd1dfc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:50.155214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1093/2177] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:50.159739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:50.187508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f3a95-94cd-4a62-b0b4-479881462639 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:50.222815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:50.223597 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:50.223923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:50.224228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] PATH_INFO: `/v3/projects/e01e1f05064247abad28bb5c5dc780e3/users/4490d9b438fc4ec5a2d96111e594b2c0/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:50.233307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01e1f05064247abad28bb5c5dc780e3, user_id=4490d9b438fc4ec5a2d96111e594b2c0, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:50.235348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:50.266436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1085/2178] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:50 2022] PUT /identity/v3/projects/e01e1f05064247abad28bb5c5dc780e3/users/4490d9b438fc4ec5a2d96111e594b2c0/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:50.278052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:50.278903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:50.279291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:50.291005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:50.316225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] MFA Rules not processed for user `4490d9b438fc4ec5a2d96111e594b2c0`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:50.317647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:50.371174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:50.373490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:50.445849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1094/2179] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:50.461534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:50.463795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:50.464006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846e86-464f-4c5b-b15e-e89f7c183e74 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:50.473227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:50.476435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:50.477242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:50.477686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] PATH_INFO: `/v3/projects/71a188048d0b4bc3a68032746259e533` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:50.480279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a188048d0b4bc3a68032746259e533)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:50.482409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:50.544421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a188048d0b4bc3a68032746259e533', 'request_id': 'req-7682b3a4-e011-4731-8d70-d619dd5c534c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:50.557887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a188048d0b4bc3a68032746259e533', 'request_id': 'req-7682b3a4-e011-4731-8d70-d619dd5c534c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:50.562119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b258f-8118-427e-b27d-a9dae6768bf9 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:50.566515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a188048d0b4bc3a68032746259e533', 'request_id': 'req-7682b3a4-e011-4731-8d70-d619dd5c534c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:50.574415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:50.579766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] The token cache is being invalidate because project 71a188048d0b4bc3a68032746259e533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:50.580045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:50.581017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:50.582788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1086/2180] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:50 2022] DELETE /identity/v3/projects/71a188048d0b4bc3a68032746259e533 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:50.631809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:50.632687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:50.632920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:50.633238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:50.637405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:50.742421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:50.745703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:50.748252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:50.778985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1095/2181] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:51.361232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:51.362642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.439411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682b3a4-e011-4731-8d70-d619dd5c534c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:51.445699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:51.446776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.447030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.450515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] PATH_INFO: `/v3/users/828614d1dc6840a19f7f2d970976ae5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.451981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828614d1dc6840a19f7f2d970976ae5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:51.453953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:51.469714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:51.471348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.488910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf2c43-6b0a-4437-8711-6c428ba5d845 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:51.495775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:51.496926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.497306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.497683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] PATH_INFO: `/v3/users/e3a030d8c10d4ecc966ca7145fece349` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.498957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a030d8c10d4ecc966ca7145fece349)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:51.501760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:51.526902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828614d1dc6840a19f7f2d970976ae5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.538217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828614d1dc6840a19f7f2d970976ae5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.539160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828614d1dc6840a19f7f2d970976ae5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.540415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828614d1dc6840a19f7f2d970976ae5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.557886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-cac5abda-8e12-4709-96aa-e6155e8a71d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828614d1dc6840a19f7f2d970976ae5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.570071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a030d8c10d4ecc966ca7145fece349'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.570564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1087/2182] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:51 2022] DELETE /identity/v3/users/828614d1dc6840a19f7f2d970976ae5d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:51.570688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a030d8c10d4ecc966ca7145fece349'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.575663 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.575951 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.576235 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.579560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a030d8c10d4ecc966ca7145fece349'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.601552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:51.601808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a030d8c10d4ecc966ca7145fece349'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.602511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2a9d79c0-e672-4010-83d9-32333294ded7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a030d8c10d4ecc966ca7145fece349'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:51.602778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.624592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1096/2183] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:51 2022] DELETE /identity/v3/users/e3a030d8c10d4ecc966ca7145fece349 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:51.628895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.629180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.629462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.654204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:51.655438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.678833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1088/2184] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:51.689949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:51.691463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.720552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:51.731373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1097/2185] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:56:51.742324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:51.743936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:51.782481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fe443-fe7f-49ad-aa94-718d983ac4f0 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:51.815328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de9b8e-22dd-46ad-8e5d-8c627ccea717 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:51.819168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:51.820768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.821189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.821531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.832956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:51.834496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:51.845312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1098/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:56:51.851727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:51.852476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:51.853089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:51.853089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:51.863666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:51.865324 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:51.876641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1089/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:56:52.295968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:52.298245 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:52.315503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af05ef-7079-4114-b53c-44a8e3225eed service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:52.321954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:52.335686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.337280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.339624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] PATH_INFO: `/v3/projects/551bc3a1de8d4b69946e0f93e6baf7c8` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.343424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551bc3a1de8d4b69946e0f93e6baf7c8)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:52.347653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:52.356821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:52.444043 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551bc3a1de8d4b69946e0f93e6baf7c8', 'request_id': 'req-369936a7-6a73-40b8-bdd3-750845392397'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.456572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551bc3a1de8d4b69946e0f93e6baf7c8', 'request_id': 'req-369936a7-6a73-40b8-bdd3-750845392397'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.465773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551bc3a1de8d4b69946e0f93e6baf7c8', 'request_id': 'req-369936a7-6a73-40b8-bdd3-750845392397'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.480615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] The token cache is being invalidate because project 551bc3a1de8d4b69946e0f93e6baf7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:52.481138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.482184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:52.483965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:52.500849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.512326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1099/2188] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:52 2022] DELETE /identity/v3/projects/551bc3a1de8d4b69946e0f93e6baf7c8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:52.517385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:52.518807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:52.527975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96547fe5-4955-47ef-94db-0a1a6ef9ee9f service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:52.640548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369936a7-6a73-40b8-bdd3-750845392397 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:52.646359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:52.647036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.647322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.647609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] PATH_INFO: `/v3/users/4490d9b438fc4ec5a2d96111e594b2c0` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.648778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4490d9b438fc4ec5a2d96111e594b2c0)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:52.651224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:52.653671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:52.654473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.654915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.655194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] PATH_INFO: `/v3/projects/22e6f064a6b34a67bd935bd3923e6829` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.656915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e6f064a6b34a67bd935bd3923e6829)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:52.659315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:52.727513 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4490d9b438fc4ec5a2d96111e594b2c0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.728314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4490d9b438fc4ec5a2d96111e594b2c0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.735762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e6f064a6b34a67bd935bd3923e6829', 'request_id': 'req-becd69f1-43a2-4bbf-abcc-7834d9b69414'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.738283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4490d9b438fc4ec5a2d96111e594b2c0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.743997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e6f064a6b34a67bd935bd3923e6829', 'request_id': 'req-becd69f1-43a2-4bbf-abcc-7834d9b69414'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.748299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4490d9b438fc4ec5a2d96111e594b2c0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.749195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-65e2c548-471e-43aa-a2e8-228e09dfc040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4490d9b438fc4ec5a2d96111e594b2c0'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.757283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e6f064a6b34a67bd935bd3923e6829', 'request_id': 'req-becd69f1-43a2-4bbf-abcc-7834d9b69414'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.778857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1100/2189] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:56:52 2022] DELETE /identity/v3/users/4490d9b438fc4ec5a2d96111e594b2c0 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:52.781179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] The token cache is being invalidate because project 22e6f064a6b34a67bd935bd3923e6829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:52.781179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.782043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-becd69f1-43a2-4bbf-abcc-7834d9b69414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:52.783600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1090/2190] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:52 2022] DELETE /identity/v3/projects/22e6f064a6b34a67bd935bd3923e6829 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:52.788669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.789431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.789905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.812670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:52.813948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:52.824828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:52.900026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1101/2191] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:52.903477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.904393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.904690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.933416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:52.935577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:52.936853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:52.947095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1091/2192] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:52.953017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:52.964440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1fbc5f-8871-49a5-9640-f044fd48a776 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:52.970087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:52.971095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:52.971385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:52.971661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:52.972628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:52.972934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:52.974809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:52.997899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1102/2193] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:53.002283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.013691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c5a84-ccfe-4676-b31f-dd5edff30a28 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.018864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.019500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.019782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.020062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.020786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.022696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.058668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1092/2194] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:53.062783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.073293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ffb000-4786-4e53-aef1-4320dbc7e391 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.120566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.122459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:53.158341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.159265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.159785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.160280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.161323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.163294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.194112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1103/2195] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:53.195338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.207585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3b6ea-8882-4d31-99de-af31ffe3f31a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.217622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.222174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.222470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.222801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.223522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.225374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.231778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b003980-65e0-483e-9e45-82b907aaf2e8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.238774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.240409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1104/2196] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:53.240570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.240570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.240570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] PATH_INFO: `/v3/projects/e01e1f05064247abad28bb5c5dc780e3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.243212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.249454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01e1f05064247abad28bb5c5dc780e3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.251674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.256372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d86da4-932d-4a8d-9e10-fd4a2225c83c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.265507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.267228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.267664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.268097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.269148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.271458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.286588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1105/2197] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:53.291878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.305809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a4eca-45a3-4ca0-ae16-2234c74de125 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.320522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01e1f05064247abad28bb5c5dc780e3', 'request_id': 'req-1ce05463-0c59-4214-8416-9ab42d3ca935'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:53.330724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01e1f05064247abad28bb5c5dc780e3', 'request_id': 'req-1ce05463-0c59-4214-8416-9ab42d3ca935'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:53.361700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01e1f05064247abad28bb5c5dc780e3', 'request_id': 'req-1ce05463-0c59-4214-8416-9ab42d3ca935'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:53.383268 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] The token cache is being invalidate because project e01e1f05064247abad28bb5c5dc780e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:56:53.383708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:53.384287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1ce05463-0c59-4214-8416-9ab42d3ca935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:56:53.386448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1093/2198] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:56:53 2022] DELETE /identity/v3/projects/e01e1f05064247abad28bb5c5dc780e3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:56:53.392491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913f96-1645-4e88-b23b-6377ff225079 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.392823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913f96-1645-4e88-b23b-6377ff225079 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.393187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d913f96-1645-4e88-b23b-6377ff225079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.407344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.408221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.409085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.409368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] PATH_INFO: `/v3/projects/a5dee6a2bfb44b53924cf6d76c5fba12/users/6398ec15831f40bfb1b316070a78adba/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.426066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:53.431326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5dee6a2bfb44b53924cf6d76c5fba12, user_id=6398ec15831f40bfb1b316070a78adba, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.435456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c59da23-dc46-4b59-bae6-4b60a4e4a36a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.446076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6d913f96-1645-4e88-b23b-6377ff225079 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:53.448736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6d913f96-1645-4e88-b23b-6377ff225079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:53.464491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1106/2199] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:53 2022] PUT /identity/v3/projects/a5dee6a2bfb44b53924cf6d76c5fba12/users/6398ec15831f40bfb1b316070a78adba/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:53.470294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.470824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.471092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.503491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:53.505111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:53.555375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1094/2200] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:53.564689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.564998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.568751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.604924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:53.607116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:53.622530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1095/2201] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:53.623911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.630403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:53.641661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322c6f2-7aa2-4182-a229-bcd96a41ae46 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.658838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.660934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1107/2202] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:53.661588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.661974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.662331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.663690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:56:53.664067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.666221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.670457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.671966 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:53.680588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1096/2203] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:56:53.684905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.701241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee7c4b-c985-48eb-a511-e2ef04ad6272 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.707516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.708317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.708627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.708908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.709742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.711674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.762972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1097/2204] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:53.764074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.803274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cd882-7109-4c91-a5d6-e4a17e29779d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.888772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.889507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.889873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.890335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.894766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce09890a-b2e2-47fd-97e5-44b281f974bb admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.901575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.902375 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.902733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.903085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.903830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.904526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.906228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.906347 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.920661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1108/2205] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:56:53.939397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1098/2206] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:53 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:53.944462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:53.954095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfc36b-a97f-47cb-97c1-f3fe90b3ea26 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:53.968898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:53.969497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:53.969777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:53.970061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:53.970741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:53.972361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:53.985248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1109/2207] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:53.989097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:54.002209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0508abb-3781-434f-9aac-f5a58108cef0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:54.017257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:54.018135 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:54.018731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:54.019122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:54.019881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:54.022831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3a42e-2078-467c-9bb1-2df488ce3e5d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:54.045605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1099/2208] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:56:54.047214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:54.068237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d361e3-cf55-40ed-a83d-f73abe63d010 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:54.084866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:54.085577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:54.085860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:54.086133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] PATH_INFO: `/v3/projects/1e27894683cf41f9a3be56ba2c73c919/users/59315f35d8074e6eb9bb5814f0c2169f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:54.113646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e27894683cf41f9a3be56ba2c73c919, user_id=59315f35d8074e6eb9bb5814f0c2169f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:54.118303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:54.137367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1110/2209] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:54 2022] PUT /identity/v3/projects/1e27894683cf41f9a3be56ba2c73c919/users/59315f35d8074e6eb9bb5814f0c2169f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:54.150241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422e5-8a20-4541-97b9-989f6f1de879 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:54.151156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422e5-8a20-4541-97b9-989f6f1de879 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:54.151576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a422e5-8a20-4541-97b9-989f6f1de879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:54.202553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-48a422e5-8a20-4541-97b9-989f6f1de879 None None] MFA Rules not processed for user `59315f35d8074e6eb9bb5814f0c2169f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:54.203859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-48a422e5-8a20-4541-97b9-989f6f1de879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:54.308367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1100/2210] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:54.322091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:54.325487 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:54.391089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:54.476685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7109fb9-1a3a-4955-92dc-bc1b4a87070e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:54.482176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:54.483017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:54.483409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:54.483986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:54.495850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:54.498723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:54.533549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1111/2211] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:56:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 20:56:59.094342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.095005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.095005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.132517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] MFA Rules not processed for user `6398ec15831f40bfb1b316070a78adba`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:56:59.135439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:59.168150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:56:59.212860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1101/2212] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:56:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:56:59.217537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.219533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:56:59.320152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32e442-d97b-48f3-b502-b0858288f456 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.330456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.331112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.331401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.331682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.332400 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.334350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.367578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1112/2213] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:56:59 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:59.371978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.383424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001bdb7-3efb-4bc5-8b44-0cf03facffaa None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.455861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.456519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.456807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.457082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.457785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.459693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.482528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1102/2214] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:56:59 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:56:59.486922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.499126 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c80fe0-de07-4b45-ae7f-151b145f86f4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.503936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.504591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.504889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.505144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.505834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.507437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.522590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1113/2215] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:59.527225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.537857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a80-7771-4b5b-a4aa-f9f1af01972d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.542685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.543381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.543660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.543932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] PATH_INFO: `/v3/projects/be8305125d0d469788b3dfa44aeb50c8/users/4236ad72ab0346be8145d528bbba7bbb/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.560547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8305125d0d469788b3dfa44aeb50c8, user_id=4236ad72ab0346be8145d528bbba7bbb, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.563209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.578852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1103/2216] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:59 2022] PUT /identity/v3/projects/be8305125d0d469788b3dfa44aeb50c8/users/4236ad72ab0346be8145d528bbba7bbb/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:56:59.582624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.593693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c763c91-9623-45e0-bee7-906512594705 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.658579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.659292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.659580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.659856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.660591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.662217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f55e5e-75aa-46b7-a2a0-50f5a1661ecf admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.674329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1114/2217] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 20:56:59.678921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:56:59.689654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc700ca-e1d0-4850-b520-310e2943d2e0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:56:59.696593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:56:59.704799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:56:59.705243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:56:59.705669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] PATH_INFO: `/v3/projects/be8305125d0d469788b3dfa44aeb50c8/users/4236ad72ab0346be8145d528bbba7bbb/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:56:59.714634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8305125d0d469788b3dfa44aeb50c8, user_id=4236ad72ab0346be8145d528bbba7bbb, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:56:59.721344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:56:59.740213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1104/2218] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:56:59 2022] PUT /identity/v3/projects/be8305125d0d469788b3dfa44aeb50c8/users/4236ad72ab0346be8145d528bbba7bbb/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:57:04.798368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930517-b87d-4283-ad92-89cae4685e4b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:04.798881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930517-b87d-4283-ad92-89cae4685e4b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:04.799375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930517-b87d-4283-ad92-89cae4685e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:04.836264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-cf930517-b87d-4283-ad92-89cae4685e4b None None] MFA Rules not processed for user `4236ad72ab0346be8145d528bbba7bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:04.837425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-cf930517-b87d-4283-ad92-89cae4685e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:04.860018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:04.933922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1115/2219] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:04.948661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:04.954904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:05.034832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669f31-5dab-46ac-a9ed-b1a03732a984 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:05.046148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:05.046148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:05.046148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:05.046148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:05.055199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:05.104707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:05.106364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e070529f-07e7-4b6d-8807-4db71d5a2dde service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:05.115968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1105/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:57:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:57:06.247032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:06.247540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:06.247646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:06.272146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] MFA Rules not processed for user `6398ec15831f40bfb1b316070a78adba`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:06.274069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:06.283871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1116/2221] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:13.943859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7009-85e8-4999-84eb-7243838e6849 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:13.944444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7009-85e8-4999-84eb-7243838e6849 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:13.944444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f7009-85e8-4999-84eb-7243838e6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:13.965274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-718f7009-85e8-4999-84eb-7243838e6849 None None] MFA Rules not processed for user `5026c90f0e3d49669c335ef542280409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:13.966503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-718f7009-85e8-4999-84eb-7243838e6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:14.024996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1106/2222] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:16.265408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=24 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:24.017615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=46 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:26.642332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:26.707370 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20acb463-0126-469d-aa67-38bbcc439abd None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:26.712086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:26.712769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:26.713052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:26.713328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] PATH_INFO: `/v3/users/59315f35d8074e6eb9bb5814f0c2169f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:26.714498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59315f35d8074e6eb9bb5814f0c2169f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:26.716530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:26.773561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59315f35d8074e6eb9bb5814f0c2169f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:26.774185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59315f35d8074e6eb9bb5814f0c2169f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:26.781974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59315f35d8074e6eb9bb5814f0c2169f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:26.789977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59315f35d8074e6eb9bb5814f0c2169f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:26.790660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59315f35d8074e6eb9bb5814f0c2169f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:26.808258 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1117/2223] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:57:26 2022] DELETE /identity/v3/users/59315f35d8074e6eb9bb5814f0c2169f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:57:26.815349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:26.815833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:26.815925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:26.836826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:26.838091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:26.897065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1107/2224] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:26.907610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:26.909053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:26.966572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54f83e-44f0-4353-8a17-b0d1eb988d29 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:26.970243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:26.970922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:26.971209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:26.971489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:26.980354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:26.981820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a01ea4-2186-4bba-9224-3fbe4ffe5e14 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:26.991409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1118/2225] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:57:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:57:27.258811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.260247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:27.274552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b999cca1-5a77-4bea-bf01-321888b39693 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.280132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.280921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.281720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.281720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] PATH_INFO: `/v3/projects/1e27894683cf41f9a3be56ba2c73c919` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.282650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e27894683cf41f9a3be56ba2c73c919)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.284634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.332522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e27894683cf41f9a3be56ba2c73c919', 'request_id': 'req-ba7c513b-db8e-4fd5-89ee-b6f32033617a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:27.339913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e27894683cf41f9a3be56ba2c73c919', 'request_id': 'req-ba7c513b-db8e-4fd5-89ee-b6f32033617a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:27.345791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e27894683cf41f9a3be56ba2c73c919', 'request_id': 'req-ba7c513b-db8e-4fd5-89ee-b6f32033617a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:27.356743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] The token cache is being invalidate because project 1e27894683cf41f9a3be56ba2c73c919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:57:27.357027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:27.358183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ba7c513b-db8e-4fd5-89ee-b6f32033617a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:57:27.359948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1108/2226] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:57:27 2022] DELETE /identity/v3/projects/1e27894683cf41f9a3be56ba2c73c919 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:57:27.365669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.365951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.366222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.385887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:27.387077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:27.404828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:27.449658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1119/2227] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:27.457608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.457968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.458190 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.480964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:27.482191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:27.491132 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1109/2228] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:27.495610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.506491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e183cd-fb3e-472d-9e0c-32e44fb45a34 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.511114 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.511739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.512020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.512292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.513104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:57:27.513348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.515582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.529213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1120/2229] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:57:27.533862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.543575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519f790-8eba-4891-8d68-4278b29d6ff9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.549379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.549993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.550275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.550550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.551470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.553420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.587764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1110/2230] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:57:27 2022] POST /identity/v3/projects => generated 363 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:57:27.589310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.602233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec8017-0db6-4bcc-bd06-f4255fc940e3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.677168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.677791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.678170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.678447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.679115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.681096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.703601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1121/2231] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:57:27 2022] POST /identity/v3/users => generated 282 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 20:57:27.707736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.718625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e20456-49b9-4a68-be48-c7000d4d0841 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.723223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.723829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.724115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.724426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.725111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.726704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.737312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1111/2232] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:57:27.741204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.751836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e8488-c21d-4199-81c6-38691a55ad1f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.756537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.757145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.757428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.757700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.758388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.759982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad91386-f610-452f-a981-4bfcfc14bbf0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.770806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1122/2233] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:57:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:57:27.775023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.785811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bb0e9-50ab-451a-b763-e65cc9a30736 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:27.791172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:27.791547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.791828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.792100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] PATH_INFO: `/v3/projects/ad03513104ad428aa8b68031e1e01fec/users/b94777dbdea04b57be19cfc8808b74e0/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.809611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad03513104ad428aa8b68031e1e01fec, user_id=b94777dbdea04b57be19cfc8808b74e0, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:27.812821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:27.826549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1112/2234] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:57:27 2022] PUT /identity/v3/projects/ad03513104ad428aa8b68031e1e01fec/users/b94777dbdea04b57be19cfc8808b74e0/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:57:27.838143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:27.838333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:27.838613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:27.869181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] MFA Rules not processed for user `b94777dbdea04b57be19cfc8808b74e0`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:57:27.870387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:27.945625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1123/2235] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:57:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:57:27.957130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:27.958900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:27.981403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:28.017297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800250c-3c28-473f-8275-d16b9ec24758 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:28.020914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:28.021523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:28.021803 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:28.022078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:28.030820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:28.032249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:28.040868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1113/2236] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:57:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:57:37.942892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:38.037029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=17 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:42.463714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:42.465217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:42.559238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8305bb9-ad39-4e85-9fc2-26a07883aa2f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:42.564654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:42.565301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:42.565568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:42.565892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:42.567905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:42.580229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:42.581846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:42.591566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1124/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:57:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:57:44.380370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:57:44.382676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:44.449527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ca9c9-71a3-4078-ba02-1ed9442dceb0 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:57:44.453590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:57:44.454734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:57:44.455020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:57:44.455408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:57:44.457222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:57:44.523549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:57:44.525034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:57:44.537333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1114/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:57:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:57:52.587422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=32 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:57:54.532472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=50 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:01.839623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:01.850906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e6db7c-cdfb-430a-84f8-e599ceea725b service placement] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:01.857719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:01.858351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:01.858635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:01.858910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:01.861367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:01.929516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:01.930963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50228f89-fba0-4da2-bc77-2a861e1824b7 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:01.941103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1125/2239] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:58:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:58:11.936317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=30 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:19.450232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:19.523047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00295b-0468-4033-89e2-871da5e16ac5 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:19.529790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:19.530460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:19.530739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:19.531006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] PATH_INFO: `/v3/users/b94777dbdea04b57be19cfc8808b74e0` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:19.532940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94777dbdea04b57be19cfc8808b74e0)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:19.534869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:19.593157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94777dbdea04b57be19cfc8808b74e0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:19.602632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94777dbdea04b57be19cfc8808b74e0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:19.603262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94777dbdea04b57be19cfc8808b74e0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:19.604144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94777dbdea04b57be19cfc8808b74e0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:19.616893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94777dbdea04b57be19cfc8808b74e0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:19.636512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1115/2240] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:58:19 2022] DELETE /identity/v3/users/b94777dbdea04b57be19cfc8808b74e0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:58:19.639752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:19.639752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:19.639752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:19.666111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:19.667029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:19.734110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1126/2241] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:19.745355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:19.746975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:19.820948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91903fdc-7a3e-4675-8eb0-5c3dcf668b79 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:19.825099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:19.825726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:19.826011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:19.826287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:19.837889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:19.839349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dc37c1-ee9e-44ee-a954-fee0c06896fd service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:19.850823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1116/2242] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:58:20.134180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.135640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:20.148149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3d0fc-d900-4881-a0dd-84323de60a62 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.153067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.153751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.154028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.154301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] PATH_INFO: `/v3/projects/ad03513104ad428aa8b68031e1e01fec` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.155460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad03513104ad428aa8b68031e1e01fec)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.157662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.207779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad03513104ad428aa8b68031e1e01fec', 'request_id': 'req-bef4783c-35c5-467b-a991-0119386db14f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:20.215452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad03513104ad428aa8b68031e1e01fec', 'request_id': 'req-bef4783c-35c5-467b-a991-0119386db14f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:20.221058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad03513104ad428aa8b68031e1e01fec', 'request_id': 'req-bef4783c-35c5-467b-a991-0119386db14f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:20.231706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] The token cache is being invalidate because project ad03513104ad428aa8b68031e1e01fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:58:20.231989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:20.232632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-bef4783c-35c5-467b-a991-0119386db14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:20.234152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1127/2243] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:58:20 2022] DELETE /identity/v3/projects/ad03513104ad428aa8b68031e1e01fec => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:58:20.239472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.239759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.240052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.261694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:20.262907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:20.279959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:20.329809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1117/2244] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:20.337418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.337711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.338045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.357614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:20.358827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:20.368137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1128/2245] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:20.372503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.382509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27936747-7e59-4094-ae02-ae7df6646e94 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.387153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.387945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.388238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.388540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.389239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 20:58:20.389540 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.391896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.409015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1118/2246] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 20:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 20:58:20.410103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.419285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698211c-0df7-42e3-93a8-581c71655cdd None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.424808 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.425612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.426216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.426524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.427374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.429689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.463881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1129/2247] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 20:58:20 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:58:20.465661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.477450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3811d1-f0d3-472b-b358-691d9a31bb3d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.547771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.547771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.547771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.547771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.548144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.548144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.569809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1119/2248] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 20:58:20 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 20:58:20.571023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.584405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de6ba8-5f13-4ad3-849c-bc93dd5edf4e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.590621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.591248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.591538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.591813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.592815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.594407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.606230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1130/2249] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:58:20.610241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.620281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc49e-9a24-458a-b01b-1b56c13c7777 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.625800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.626421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.626706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.626986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.627682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.629317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dfde6-ce7f-4a82-b9c2-f3eee5c8c8b1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.644031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1120/2250] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 20:58:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 20:58:20.645282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.655374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c145d265-afbb-47f3-aa48-3ab580b78cb1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.661706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.662762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.663274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.663775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] PATH_INFO: `/v3/projects/9fdebe8d54c04984b24120933e9f80a4/users/929a9cbf4289422197faca915564f669/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.680131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fdebe8d54c04984b24120933e9f80a4, user_id=929a9cbf4289422197faca915564f669, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.682827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.701274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1131/2251] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 20:58:20 2022] PUT /identity/v3/projects/9fdebe8d54c04984b24120933e9f80a4/users/929a9cbf4289422197faca915564f669/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:58:20.718066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.718466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.718782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.748747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] MFA Rules not processed for user `929a9cbf4289422197faca915564f669`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:20.750469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:20.819686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1121/2252] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 20:58:20.832203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.833946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:20.859323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:20.896144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa72c32-90b1-4421-93fc-372b8b1e55fc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:20.899824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:20.900468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:20.900753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:20.901029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:20.909868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:20.911280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206494b6-b68c-4ec2-b62f-b5a794bf490f service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:20.919986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1132/2253] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:58:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:58:20.946441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:20.965536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:21.011233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67916464-caa4-4938-a15c-0cc305066be7 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:21.016391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:21.017039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:21.017321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:21.017595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] PATH_INFO: `/v3/users/929a9cbf4289422197faca915564f669` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:21.018776 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=929a9cbf4289422197faca915564f669)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:21.020738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:21.072634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '929a9cbf4289422197faca915564f669'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.080981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '929a9cbf4289422197faca915564f669'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.081913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '929a9cbf4289422197faca915564f669'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.082824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '929a9cbf4289422197faca915564f669'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.096758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '929a9cbf4289422197faca915564f669'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.110523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1122/2254] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 20:58:20 2022] DELETE /identity/v3/users/929a9cbf4289422197faca915564f669 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 20:58:21.114939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:21.115226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:21.115503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:21.134048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:21.135240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:21.194060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1133/2255] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:21.204314 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:21.205772 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:21.264129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0105b3-e90d-4063-b302-d5a12ddbc0ce admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:21.268671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:21.269294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:21.269579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:21.269857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:21.279142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:21.280606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3add1-f1e1-41ab-9ac4-86675c5a0d7c service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:21.291427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1123/2256] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:58:21.572574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:21.574092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:21.589102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21fe11-4824-44ab-b8c5-6581a480563a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:21.594307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:21.594997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:21.595282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:21.595553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] PATH_INFO: `/v3/projects/9fdebe8d54c04984b24120933e9f80a4` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:21.596750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fdebe8d54c04984b24120933e9f80a4)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:21.598670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:21.653066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fdebe8d54c04984b24120933e9f80a4', 'request_id': 'req-704751b2-a364-4f27-a336-c4b44517a703'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.661532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fdebe8d54c04984b24120933e9f80a4', 'request_id': 'req-704751b2-a364-4f27-a336-c4b44517a703'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.667332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fdebe8d54c04984b24120933e9f80a4', 'request_id': 'req-704751b2-a364-4f27-a336-c4b44517a703'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.679143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] The token cache is being invalidate because project 9fdebe8d54c04984b24120933e9f80a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 20:58:21.679429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.680069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-704751b2-a364-4f27-a336-c4b44517a703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 20:58:21.681603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1134/2257] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 20:58:21 2022] DELETE /identity/v3/projects/9fdebe8d54c04984b24120933e9f80a4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 20:58:31.287030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:31.676138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:33.819148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:33.819617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:33.819727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:33.839302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] MFA Rules not processed for user `807855d862b146bdba009626f28d8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:33.840628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:33.894921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1124/2258] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:58:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:43.887213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=25 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:49.610476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee01bc-cccf-4b0a-9149-8cbaf00b0598 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:49.611018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee01bc-cccf-4b0a-9149-8cbaf00b0598 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:49.611018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee01bc-cccf-4b0a-9149-8cbaf00b0598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:49.630996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-3dee01bc-cccf-4b0a-9149-8cbaf00b0598 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:58:49.632217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3dee01bc-cccf-4b0a-9149-8cbaf00b0598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:49.693268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1135/2259] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:58:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:58:49.735700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:58:49.737166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:58:49.810433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501c20f-d778-4edd-96e0-cc2a7282dfb6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:58:49.817335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:58:49.817967 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:58:49.818251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:58:49.818527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:58:49.826949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:58:49.829748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40182861-8765-44ce-aca4-c7257fbc85ca service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:58:49.837898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1125/2260] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 20:58:59.686399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=25 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:58:59.834588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=32 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:04.890200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:04.890803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:04.890803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:04.892154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1136/2261] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:59:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:59:04.898363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821c6f-78cd-47e9-ada2-ad6e00c8bf95 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:04.898833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821c6f-78cd-47e9-ada2-ad6e00c8bf95 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:04.898946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88821c6f-78cd-47e9-ada2-ad6e00c8bf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:04.919788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-88821c6f-78cd-47e9-ada2-ad6e00c8bf95 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:04.921194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-88821c6f-78cd-47e9-ada2-ad6e00c8bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:04.991537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1126/2262] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:59:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:59:05.004164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:05.005881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:05.064119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8563b-6186-499f-b1bc-e06c97971acc None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:05.067559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:05.068197 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.068510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.068784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.079237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:05.080678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818ac82-0d24-4628-9a74-07c0e8f371f5 service glance] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:05.090983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1137/2263] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:05.159148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.159436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.159716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] PATH_INFO: `/` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.161148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1127/2264] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:59:05.165975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1ef3-8cf9-4727-9d8a-f51d09a9cc11 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.166261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1ef3-8cf9-4727-9d8a-f51d09a9cc11 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.166536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e1ef3-8cf9-4727-9d8a-f51d09a9cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.187540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2b4e1ef3-8cf9-4727-9d8a-f51d09a9cc11 None None] MFA Rules not processed for user `9ef5ca1eeba741259394f0ab37dd3751`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:05.188768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2b4e1ef3-8cf9-4727-9d8a-f51d09a9cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:05.239910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1138/2265] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:59:05.252291 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:05.254043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:05.267208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d260d-8dfc-447d-a01c-4aa8646ed372 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:05.270695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:05.271297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.271578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.271850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.279193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:05.280682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1e1b8-5ccd-42bb-9d6d-2499f850f451 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:05.289059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1128/2266] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:05.371818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca277da-049b-48dc-8234-8309289a4ba5 None None] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.372127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca277da-049b-48dc-8234-8309289a4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.372422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca277da-049b-48dc-8234-8309289a4ba5 None None] PATH_INFO: `/` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.373904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1139/2267] 10.222.0.52 () {54 vars in 921 bytes} [Thu Aug 4 20:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 20:59:05.379498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dabfc-f954-44d7-a5e3-1f5b1053b409 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.379791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dabfc-f954-44d7-a5e3-1f5b1053b409 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.380070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136dabfc-f954-44d7-a5e3-1f5b1053b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.400193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-136dabfc-f954-44d7-a5e3-1f5b1053b409 None None] MFA Rules not processed for user `02a04385570944ecbcaf96003f5c5753`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:05.401481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-136dabfc-f954-44d7-a5e3-1f5b1053b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:05.411625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1129/2268] 10.222.0.52 () {58 vars in 1018 bytes} [Thu Aug 4 20:59:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:59:05.421945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca277da-049b-48dc-8234-8309289a4ba5 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:05.432921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca277da-049b-48dc-8234-8309289a4ba5 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:05.436499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:05.437109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:05.437388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:05.437662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:05.447657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:05.449123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:05.460408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1140/2269] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:08.335072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:08.335538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:08.335649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:08.359754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:08.361523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:08.371874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1130/2270] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 20:59:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:59:08.384672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:08.396028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302517ee-3df3-49f1-9286-600df347b67e service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:08.401199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:08.401807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:08.402117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:08.402389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:08.411815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:08.413597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:08.423632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1141/2271] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:59:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:59:18.352463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=43 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:18.418910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=61 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:23.907313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:23.916482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ab020-8b9e-45ad-abf9-e86453ce9742 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:23.919969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:23.920685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:23.920966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:23.921237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:23.922930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:23.936482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:23.937903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:23.948611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1131/2272] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:26.364081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:26.373288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fc6a7-341d-4c46-8083-9895b002fb4a service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:26.377044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:26.377649 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:26.377935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:26.378209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:26.379987 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:26.394666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:26.396081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d0c5f-9012-42fb-a8c6-2ec57e1d0908 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:26.407936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1142/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:33.943821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:36.402871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:40.610638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:40.612191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:40.626351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4c4f1-7157-47a3-9ff0-95c572367590 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:40.631833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:40.632467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:40.632748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:40.633024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:40.634763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:40.693040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:40.694636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:40.704043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1132/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:41.461611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:41.462104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:41.462225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:41.481431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:41.483170 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:41.490442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1143/2275] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 20:59:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 20:59:41.503487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:41.515325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc679cd-8c7e-4c79-bf06-627fa84a1cd2 service nova] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:41.520265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:41.520929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:41.521213 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:41.521490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:41.531129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:41.532761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:41.540782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1133/2276] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 20:59:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 20:59:42.166060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:42.178836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234a07e-bfdb-463a-8e80-de8c7bd07f60 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:42.184177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:42.184803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:42.185113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:42.185451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:42.187175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:42.249106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:42.250746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:42.260084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1144/2277] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:51.537629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=34 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:52.251717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:52.255600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=31 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 20:59:52.263747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6a959-b905-41b1-b5fc-5caa82b3f148 service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:52.269122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:52.269728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:52.270013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:52.270292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:52.272017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:52.333055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:52.334582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120598e-f497-478f-a65e-ff157f69de34 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:52.344989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1134/2278] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:55.229813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 20:59:55.239473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e65979-31a6-4799-9a7a-046d3a303e4c service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 20:59:55.243048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 20:59:55.243667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:55.243953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:55.244233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:55.246055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:55.266706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 20:59:55.268314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 20:59:55.277678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1145/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 20:59:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 20:59:57.728861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 20:59:57.729348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 20:59:57.729477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 20:59:57.751112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] MFA Rules not processed for user `5026c90f0e3d49669c335ef542280409`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 20:59:57.753344 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 20:59:57.760749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1135/2280] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 20:59:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:00:05.274408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:00:07.743797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=31 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:00:33.429438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:00:33.442395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206444-1b9e-40ff-b723-3ee9923c4404 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:00:33.448509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:00:33.449136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:33.449424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:33.449746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:33.451809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:33.462433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:00:33.464004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b071ed3-8eff-41a9-99df-2f61b37126fa service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:00:33.473622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1146/2281] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:00:37.171448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:00:37.186378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce29ac2-5753-4552-bd9f-9dda2a0cc70b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:00:37.192150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:00:37.193048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:37.193594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:37.194079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:37.195961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:37.210150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:00:37.211906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:00:37.223826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1136/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:00:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:00:43.469482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:00:47.219188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:00:48.802543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:48.803056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:48.803149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:48.825684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] MFA Rules not processed for user `807855d862b146bdba009626f28d8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:00:48.826928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:48.836148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1147/2283] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:00:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:00:49.217975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:00:49.226928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786916f5-a50e-4fcb-9aee-0753231a028a service nova] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:00:49.230523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:00:49.231936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:49.232425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:49.232712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:49.234516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:49.243969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:00:49.245410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:00:49.253517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1137/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:00:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:00:50.586930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:00:50.588719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:50.670630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5452e4-300c-45a9-9cb7-d68a38e3ff0f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:00:50.676482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:00:50.677111 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:50.677462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:50.677738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:50.679509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:50.693206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:00:50.695420 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:00:50.711338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1148/2285] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:00:54.339397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:00:54.368853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10418c9-2406-4a23-a272-92067c15de44 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:00:54.378701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:00:54.379573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:00:54.380032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:00:54.380303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:00:54.382483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:00:54.398914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:00:54.400789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c6d66-7039-44fa-891d-21efe24eae3a service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:00:54.417395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1138/2286] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:00:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:00.704439 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=35 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:01.048489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:01.050857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:01.071406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939305a-68fb-405a-a122-e6cca1e232b2 service placement] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:01.082960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:01.083704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:01.089209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:01.089706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:01.092304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:01.111720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:01.114603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:01.130557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1149/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:04.409864 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=12 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:04.672316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccce1-64e1-480f-bb3e-2dac993c0d8f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:04.672638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccce1-64e1-480f-bb3e-2dac993c0d8f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:04.672921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccce1-64e1-480f-bb3e-2dac993c0d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:04.692026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3a8ccce1-64e1-480f-bb3e-2dac993c0d8f None None] MFA Rules not processed for user `a1930e444b5b47ea8e606c7ee4da600f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:04.693198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3a8ccce1-64e1-480f-bb3e-2dac993c0d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:04.700952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1139/2288] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:05.092823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:05.102351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f28c7-8a25-408f-8045-930352f5eb6d service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:05.106539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:05.107319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:05.107595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:05.107863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:05.109664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:05.120849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:05.122404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:05.130941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1150/2289] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:12.341262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:12.341262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:12.341262 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:12.362042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:12.363370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:12.371908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1140/2290] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:12.454557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:12.472215 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d564f14-1b56-4624-b878-862f4932a6b3 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:12.478886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:12.479849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:12.480304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:12.480818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:12.493802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:12.496635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:12.506781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1151/2291] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:01:17.484226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:17.493859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0bac2-e6b2-4495-a6d7-4bbbd8962f41 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:17.498680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:17.499294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:17.499577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:17.499854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:17.501758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:17.566621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:17.568056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:17.578151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1141/2292] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:22.503835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=18 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:22.640405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:22.641845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:22.656454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b27c76-2f42-40d5-a61a-0000674fcb67 service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:22.660961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:22.661602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:22.661920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:22.662200 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] PATH_INFO: `/v3/users/c10b35e8364c4789b7a926888f1e7234` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:22.663672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10b35e8364c4789b7a926888f1e7234)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:22.665629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:22.713958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10b35e8364c4789b7a926888f1e7234'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:22.715131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10b35e8364c4789b7a926888f1e7234'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:22.723718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10b35e8364c4789b7a926888f1e7234'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:22.735210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10b35e8364c4789b7a926888f1e7234'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:22.735898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10b35e8364c4789b7a926888f1e7234'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:22.754326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1152/2293] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:22 2022] DELETE /identity/v3/users/c10b35e8364c4789b7a926888f1e7234 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:22.947573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:22.958590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb59055-246f-46d8-ac31-17a689fce77a service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.020452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.021119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.021404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.021908 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] PATH_INFO: `/v3/projects/06b449da3f4248579c6a12f07c7f5b55` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.023372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b449da3f4248579c6a12f07c7f5b55)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.025789 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.081732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b449da3f4248579c6a12f07c7f5b55', 'request_id': 'req-493f4df1-6684-41ad-959f-e455f0a89e88'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.090226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b449da3f4248579c6a12f07c7f5b55', 'request_id': 'req-493f4df1-6684-41ad-959f-e455f0a89e88'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.096237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b449da3f4248579c6a12f07c7f5b55', 'request_id': 'req-493f4df1-6684-41ad-959f-e455f0a89e88'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.108936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] The token cache is being invalidate because project 06b449da3f4248579c6a12f07c7f5b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:23.109241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.109865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.111450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1142/2294] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:22 2022] DELETE /identity/v3/projects/06b449da3f4248579c6a12f07c7f5b55 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:23.115834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.117538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:23.131735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c757c5cf-1ed9-4c31-ac5a-6e6f66601fad admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.191053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.191730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.192013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.192292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] PATH_INFO: `/v3/users/5322d6d8a0d54487bb7f3ff30f4c7f8f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.193472 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5322d6d8a0d54487bb7f3ff30f4c7f8f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.195403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.240903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5322d6d8a0d54487bb7f3ff30f4c7f8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.241489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5322d6d8a0d54487bb7f3ff30f4c7f8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.248873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5322d6d8a0d54487bb7f3ff30f4c7f8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.255633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5322d6d8a0d54487bb7f3ff30f4c7f8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.256257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0ae42684-95b3-44c4-b0da-dff64c55a4f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5322d6d8a0d54487bb7f3ff30f4c7f8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.264695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:23.276311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1153/2295] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:23 2022] DELETE /identity/v3/users/5322d6d8a0d54487bb7f3ff30f4c7f8f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:23.456208 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.458500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:23.476843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f4df1-6684-41ad-959f-e455f0a89e88 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.488038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:23.546305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.546972 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.547258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.547530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] PATH_INFO: `/v3/projects/762735c015754691b9dd41f33bde68c0` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.548733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] RBAC: Authorizing `identity:delete_project(project_id=762735c015754691b9dd41f33bde68c0)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.550868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.597219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762735c015754691b9dd41f33bde68c0', 'request_id': 'req-805abc0c-a21b-4976-806e-0f80869496df'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.604130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762735c015754691b9dd41f33bde68c0', 'request_id': 'req-805abc0c-a21b-4976-806e-0f80869496df'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.610179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762735c015754691b9dd41f33bde68c0', 'request_id': 'req-805abc0c-a21b-4976-806e-0f80869496df'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.621794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] The token cache is being invalidate because project 762735c015754691b9dd41f33bde68c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:23.622075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.623097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-805abc0c-a21b-4976-806e-0f80869496df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:23.624184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1143/2296] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:23 2022] DELETE /identity/v3/projects/762735c015754691b9dd41f33bde68c0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:23.629005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.629453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.629550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.651333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:23.652679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:23.720797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1154/2297] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:01:23.722387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.722680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.722959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.743046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:23.744182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:23.753129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1144/2298] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:01:23.757459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.769137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f640acd-1958-4eb4-b742-3326e8347caa None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.773931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.774550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.774833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.775106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.775801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:23.776146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.777917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.792372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1155/2299] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:01:23.794921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.804607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a17630-b70c-4481-b161-4660245a7a3e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.809851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.810464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.810749 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.811027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.811689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.813646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.845365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1145/2300] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:23 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:23.851042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.861253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbcf81-bddf-45dc-9cf2-6c94ae8e5181 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.921217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.921840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.922097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.922531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.923210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:23.925613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:23.970072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1156/2301] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:23 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:01:23.976123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:23.989220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b274558f-16fd-4548-a83e-4238c2c629cc admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:23.994381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:23.994986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:23.995460 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:23.997711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:23.998462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:24.000092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:24.012665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1146/2302] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:24.021360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:24.037938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5278baf-0bf5-4b86-84ea-57b08de9d49a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:24.047416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:24.048153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:24.048510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:24.048787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:24.049510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:24.051380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea601704-cc5e-4454-b2cf-73bd0d64411b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:24.066493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1157/2303] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:24.068018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:24.078997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d8a372-73d4-4eb1-b980-1c6369eb26af admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:24.083921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:24.084821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:24.085095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:24.085343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] PATH_INFO: `/v3/projects/7535eb1709bc4bfe8a1e251fb0db4593/users/3f4c9c7f2ddd4403b13696794dfaab5d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:24.102180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7535eb1709bc4bfe8a1e251fb0db4593, user_id=3f4c9c7f2ddd4403b13696794dfaab5d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:24.105842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:24.126040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1147/2304] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:24 2022] PUT /identity/v3/projects/7535eb1709bc4bfe8a1e251fb0db4593/users/3f4c9c7f2ddd4403b13696794dfaab5d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:24.132388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26cf4-da23-48f1-8690-a2acb61bdf4b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:24.132713 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26cf4-da23-48f1-8690-a2acb61bdf4b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:24.132996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b26cf4-da23-48f1-8690-a2acb61bdf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:24.154304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-29b26cf4-da23-48f1-8690-a2acb61bdf4b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:24.155531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-29b26cf4-da23-48f1-8690-a2acb61bdf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:24.222185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:24.237875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1158/2305] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:24.248887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:24.250649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:24.307615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:24.326539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc9582-b07e-45bc-88dd-329dd618f437 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:24.330318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:24.330953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:24.331236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:24.331511 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:24.342868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:24.344372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:24.357254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1148/2306] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:01:29.332537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.333026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.333122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.365373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] MFA Rules not processed for user `3f4c9c7f2ddd4403b13696794dfaab5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:29.366589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:29.468801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1159/2307] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:29.472879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.492299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8471c-dcbc-403c-b174-ab2f0813e02f service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.509287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.510110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.510463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.510753 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.511985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.514005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.545503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1149/2308] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:29 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:29.549712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.564139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8158b7e-bb47-402e-9eb4-d8b3f00a6b66 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.630014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.630635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.630922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.631206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.631888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.633850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.655490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1160/2309] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:29 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:29.659625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.670251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9849e2-9f9a-437e-ab0a-b834c9eb6627 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.675086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.675958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.676237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.676545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.677233 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.678857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.691202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1150/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:29.695859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.707404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7455cde-780d-4307-b104-22c16f65e12e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.711941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.712810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.713090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.713366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] PATH_INFO: `/v3/projects/91f8baeb2a84415f8b767cf708fcbf64/users/48e10d09dfc941dc93e7a38228a0114e/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.728253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f8baeb2a84415f8b767cf708fcbf64, user_id=48e10d09dfc941dc93e7a38228a0114e, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.730904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.743474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1161/2311] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:29 2022] PUT /identity/v3/projects/91f8baeb2a84415f8b767cf708fcbf64/users/48e10d09dfc941dc93e7a38228a0114e/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:29.748194 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.759129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b8a012-0b6e-4379-8294-194f64627d7a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.820673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.821294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.821578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.821852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.822541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.825305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.835126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1151/2312] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:29.840210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:29.851151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbdcc72-6309-4e75-b59d-e5deed7c5bad admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:29.855815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:29.856500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:29.856779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:29.857050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] PATH_INFO: `/v3/projects/91f8baeb2a84415f8b767cf708fcbf64/users/48e10d09dfc941dc93e7a38228a0114e/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:29.858976 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f8baeb2a84415f8b767cf708fcbf64, user_id=48e10d09dfc941dc93e7a38228a0114e, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:29.861633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bb1d6-a3a0-4238-87fb-678b9032bfa9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:29.872738 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1162/2313] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:29 2022] PUT /identity/v3/projects/91f8baeb2a84415f8b767cf708fcbf64/users/48e10d09dfc941dc93e7a38228a0114e/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:34.663363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:34.669900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:34.832732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56929d6-a5ea-49de-b3f6-0350a4c4311b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:34.964149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:34.965514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:34.965800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:34.966080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:34.971182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:34.991509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:35.084680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:35.086909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:35.099754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1152/2314] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:36.809279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d48b-11d6-4f02-89f6-5fc74c76673a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:36.809747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d48b-11d6-4f02-89f6-5fc74c76673a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:36.810017 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466d48b-11d6-4f02-89f6-5fc74c76673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:36.850770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a466d48b-11d6-4f02-89f6-5fc74c76673a None None] MFA Rules not processed for user `48e10d09dfc941dc93e7a38228a0114e`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:36.852447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a466d48b-11d6-4f02-89f6-5fc74c76673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:36.892677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:36.953625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1163/2315] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:36.968887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:36.970304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:37.069196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da158b6-ac31-4168-95e4-b8daee0efc63 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:37.076448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:37.077104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:37.077525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:37.077804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:37.160944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:37.162854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:37.189963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1153/2316] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:39.670447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:39.670989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:39.670989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:39.691532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] MFA Rules not processed for user `87929879b1c74efea0a390a3ce1a4006`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:39.692887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:39.755744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1164/2317] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:41.313826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:41.315582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:41.345529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:41.351180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:41.378863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947076-7216-4261-9dd4-adc0c33b428f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:41.384386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:41.385310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:41.386432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:41.387343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] PATH_INFO: `/v3/users/5026c90f0e3d49669c335ef542280409` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:41.390504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5026c90f0e3d49669c335ef542280409)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:41.393619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:41.420176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:41.461610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7e644-a52c-406a-90c0-94f997d4357f service nova] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:41.470121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5026c90f0e3d49669c335ef542280409'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.470829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5026c90f0e3d49669c335ef542280409'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.473782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:41.481230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:41.481538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:41.481822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:41.483834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:41.491474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5026c90f0e3d49669c335ef542280409'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.499476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:41.503246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5026c90f0e3d49669c335ef542280409'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.504286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95e305b3-0f84-4e14-abbd-e95b48f5aaf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5026c90f0e3d49669c335ef542280409'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.506823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:41.531526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1154/2318] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:41.547686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1165/2319] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:41 2022] DELETE /identity/v3/users/5026c90f0e3d49669c335ef542280409 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:41.764383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:41.765996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:41.788019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0603a352-e962-46c6-ac8c-6fb9e0fb7d0c service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:41.878219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:41.878903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:41.879188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:41.879469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] PATH_INFO: `/v3/projects/71b3821ed2f44e848959ac9d2683dab6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:41.880775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b3821ed2f44e848959ac9d2683dab6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:41.883494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:41.936861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b3821ed2f44e848959ac9d2683dab6', 'request_id': 'req-96216017-22ee-463d-9d7e-1aa68915b1c7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.946590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b3821ed2f44e848959ac9d2683dab6', 'request_id': 'req-96216017-22ee-463d-9d7e-1aa68915b1c7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.953371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b3821ed2f44e848959ac9d2683dab6', 'request_id': 'req-96216017-22ee-463d-9d7e-1aa68915b1c7'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.965874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] The token cache is being invalidate because project 71b3821ed2f44e848959ac9d2683dab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:41.966099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.966760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-96216017-22ee-463d-9d7e-1aa68915b1c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:41.968288 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1155/2320] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:41 2022] DELETE /identity/v3/projects/71b3821ed2f44e848959ac9d2683dab6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:41.973276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:41.973600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:41.973894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:41.997998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:41.999373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:42.050886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:42.075629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1166/2321] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:42.084101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.084440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.084693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.109545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:42.110761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:42.119083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1156/2322] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:42.123297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.136047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb27842-3b58-4909-927f-d8a9ff4ec273 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.143173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.143944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.144312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.144705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.145484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:42.145855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.147846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.181076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1167/2323] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:42.193316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.204656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13368ec8-46cb-4b70-927d-0f6f9860ce93 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.212869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.214284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.214810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.215185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.216031 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.218028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.266378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1157/2324] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:42 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:42.267085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.280634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5858cb-ca25-4b8d-828e-6ec56954a679 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.362644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.363360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.363624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.363972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.364687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.368814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.401722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.414871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1168/2325] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:42 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:01:42.419956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ff835-b70c-48db-b66b-a0e2cee17ca8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.425721 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.426459 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.426746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.427284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.428060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.429853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.453935 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.463795 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1158/2326] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:42.470071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38833a69-8e2b-41c4-a98d-9314156a42d8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.487896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.488568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.488855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.489132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.489980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.501559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fcc846-e31e-4f59-a5a0-aa8568a6c233 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.538506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1169/2327] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:01:42.539701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.551653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399d887-cda6-4fe6-8e23-2971ed442807 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.558292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.563685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.563685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.563685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] PATH_INFO: `/v3/projects/969715bf5eeb4eb3b893237635545c33/users/7404b899f870415cbf64b6c9d5403632/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.579312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=969715bf5eeb4eb3b893237635545c33, user_id=7404b899f870415cbf64b6c9d5403632, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.582340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.600735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1159/2328] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:42 2022] PUT /identity/v3/projects/969715bf5eeb4eb3b893237635545c33/users/7404b899f870415cbf64b6c9d5403632/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:42.614012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.614369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.614570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.667262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] MFA Rules not processed for user `7404b899f870415cbf64b6c9d5403632`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:42.668554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:42.770744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1170/2329] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:42.785112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:42.787021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:42.792919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:42.917071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84273e7d-78d5-42a0-92e4-811c49b67e9b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:42.932040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:42.934965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:42.935408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:42.935893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:42.950035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:42.951666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d635410-8dc9-4e36-9f79-5491f5479148 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:42.964409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1160/2330] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:43.282566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:43.337158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:43.370372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5b4f9-f210-4c83-b701-4bd19dd606d3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:43.376956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:43.378793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:43.379231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:43.380048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] PATH_INFO: `/v3/users/7404b899f870415cbf64b6c9d5403632` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:43.381322 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7404b899f870415cbf64b6c9d5403632)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:43.384105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:43.478588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7404b899f870415cbf64b6c9d5403632'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:43.480249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7404b899f870415cbf64b6c9d5403632'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:43.490025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7404b899f870415cbf64b6c9d5403632'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:43.506121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7404b899f870415cbf64b6c9d5403632'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:43.506934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7404b899f870415cbf64b6c9d5403632'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:43.532411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1171/2331] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:43 2022] DELETE /identity/v3/users/7404b899f870415cbf64b6c9d5403632 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:43.534212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:43.536884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:43.537164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:43.567976 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:43.569270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:43.648474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1161/2332] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:43.663298 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:43.664984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:43.753153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba095006-0dff-4cc8-b02f-2c952a5a6841 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:43.758328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:43.759087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:43.759375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:43.759651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:43.771015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:43.772665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a66d9-ed83-46c5-ad84-f8c0eecac11f service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:43.794280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1172/2333] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:44.140633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.142410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:44.156850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70fb58-7203-421b-982e-1fb4b7cf0f93 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.177071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.177984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.178392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.179099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] PATH_INFO: `/v3/projects/969715bf5eeb4eb3b893237635545c33` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.180409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=969715bf5eeb4eb3b893237635545c33)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.183267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.277579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '969715bf5eeb4eb3b893237635545c33', 'request_id': 'req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:44.302281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '969715bf5eeb4eb3b893237635545c33', 'request_id': 'req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:44.316319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '969715bf5eeb4eb3b893237635545c33', 'request_id': 'req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:44.341547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] The token cache is being invalidate because project 969715bf5eeb4eb3b893237635545c33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:44.341996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:44.344223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4e8f1e0a-61f1-4549-8e90-5aa3f18878e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:44.381079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.381666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.381922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.396644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1162/2334] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:44 2022] DELETE /identity/v3/projects/969715bf5eeb4eb3b893237635545c33 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:44.420300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:44.422462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:44.560459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:44.584570 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1173/2335] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:44.592564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.593109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.593109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.642414 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:44.643653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:44.660725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1163/2336] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:44.661141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.672369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b1038-fa99-4603-9a9e-a575461cb393 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.677778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.678488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.678704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.678947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.679654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:44.680238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.682773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.707703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1174/2337] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:44.708941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.714988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:44.719139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79fd1e0-2280-499b-81a1-28c467ee2b3e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.726048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.726784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.727249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.727680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.728585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.730905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.770274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1164/2338] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:44 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:44.771405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.783255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bcdf92-3708-4069-a786-0ec0d48a6ac1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.861302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.861931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.862212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.862482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.863145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.865157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.895140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1175/2339] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:44 2022] POST /identity/v3/users => generated 302 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:44.899305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.910748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0745ba6a-df8f-43df-bb89-5d2d3a195654 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.918193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.918806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.919142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.920707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.921378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.923106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.939206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1165/2340] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:44.940508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.952158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60536ea-3d19-4635-86ce-fd62ed9c3023 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:44.957259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:44.957886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:44.958172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:44.958495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:44.959382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:44.961104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94112a6-56de-4cf1-98f1-c7bf8fdd5656 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:44.973445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1176/2341] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:44.977855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:44.994836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141727d9-2233-4ad0-b808-f2f78e5d1a9d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.001394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.002180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.002539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.002893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] PATH_INFO: `/v3/projects/ccb7c8dc8f6c4a1c9811ac13025539ba/users/7514eb38c7664345a542e245b70aafab/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.019340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb7c8dc8f6c4a1c9811ac13025539ba, user_id=7514eb38c7664345a542e245b70aafab, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.022053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.044046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1166/2342] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:44 2022] PUT /identity/v3/projects/ccb7c8dc8f6c4a1c9811ac13025539ba/users/7514eb38c7664345a542e245b70aafab/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:45.051675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.051979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.052260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.089809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] MFA Rules not processed for user `7514eb38c7664345a542e245b70aafab`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:45.091076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:45.173226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1177/2343] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:45.179296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.245824 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cfd29e-26d8-4d81-8b74-b5182459f5ab admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.250840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.251467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.251754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.252026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.252731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.254640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.283626 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1167/2344] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:45 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:01:45.287669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.303377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae803e-42bc-419d-8c4a-8e9b74f731c4 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.357175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:45.368022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.368798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.369311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.369825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.370871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.373068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.396518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1178/2345] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:45 2022] POST /identity/v3/users => generated 309 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:45.400587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.411605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe74906-a967-4ee3-b7cd-b576743d0111 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.417333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.417984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.418355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.418635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.419333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.421053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.433039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1168/2346] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:01:45.437180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.447074 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d03cf3-ae4f-4f44-bbe6-9b89fa740096 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.451788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.452484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.452776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.453053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] PATH_INFO: `/v3/projects/ec15397431774f96a0e5f1d749ccd19a/users/1fb25f24d1a7419c9e54046691f24d7c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.470196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec15397431774f96a0e5f1d749ccd19a, user_id=1fb25f24d1a7419c9e54046691f24d7c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.472941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.486955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1179/2347] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:45 2022] PUT /identity/v3/projects/ec15397431774f96a0e5f1d749ccd19a/users/1fb25f24d1a7419c9e54046691f24d7c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:45.490968 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.501158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff3862-b441-439a-bad6-f7c86a979cab admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.560495 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.561160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.561447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.561729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.562447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.564087 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167333f-cfda-4213-9ae1-0fbfa908f235 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.575012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1169/2348] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:45.579019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.590001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1cda3-549d-4bf9-aca3-d1332882f49c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.594633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.595288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.595569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.595847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] PATH_INFO: `/v3/projects/ec15397431774f96a0e5f1d749ccd19a/users/1fb25f24d1a7419c9e54046691f24d7c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.597794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec15397431774f96a0e5f1d749ccd19a, user_id=1fb25f24d1a7419c9e54046691f24d7c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.600475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.610989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1180/2349] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:45 2022] PUT /identity/v3/projects/ec15397431774f96a0e5f1d749ccd19a/users/1fb25f24d1a7419c9e54046691f24d7c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:45.626069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.626069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.626069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.658264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] MFA Rules not processed for user `1fb25f24d1a7419c9e54046691f24d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:45.659524 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:45.670080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:45.743907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1170/2350] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:45.755545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:45.757279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:45.825533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f14115-6352-4a81-95f8-5725c457ed16 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:45.829156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:45.829779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:45.830063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:45.830338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:45.839456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:45.840912 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e5c01c-3b3b-433e-a579-ab7a784502fc service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:45.850211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1181/2351] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:46.011026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:46.077046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4bd7-001a-4dc5-86ab-40476ce71e6e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:46.081642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:46.082317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:46.082599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:46.082876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] PATH_INFO: `/v3/users/7514eb38c7664345a542e245b70aafab` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:46.084028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7514eb38c7664345a542e245b70aafab)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:46.086024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:46.137717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7514eb38c7664345a542e245b70aafab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.144900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7514eb38c7664345a542e245b70aafab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.145571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7514eb38c7664345a542e245b70aafab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.146341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7514eb38c7664345a542e245b70aafab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.159017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7514eb38c7664345a542e245b70aafab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.178223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1171/2352] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:46 2022] DELETE /identity/v3/users/7514eb38c7664345a542e245b70aafab => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:46.183057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:46.183352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:46.183630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:46.203703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:46.204988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:46.268384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1182/2353] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:46.279384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:46.280932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:46.310960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:46.346331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dd67f-f4cf-4ae6-a5c0-f5e24051c6e4 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:46.350281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:46.351011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:46.351343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:46.351624 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:46.369088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:46.370646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:46.380645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1172/2354] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:46.667545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:46.669119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:46.681330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2264ef-8030-4c7c-9603-2f403446a9ab None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:46.686180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:46.686865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:46.687147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:46.687428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] PATH_INFO: `/v3/projects/ccb7c8dc8f6c4a1c9811ac13025539ba` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:46.688614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb7c8dc8f6c4a1c9811ac13025539ba)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:46.690525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:46.739714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:46.742997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb7c8dc8f6c4a1c9811ac13025539ba', 'request_id': 'req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.751037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb7c8dc8f6c4a1c9811ac13025539ba', 'request_id': 'req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.756716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb7c8dc8f6c4a1c9811ac13025539ba', 'request_id': 'req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.769499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] The token cache is being invalidate because project ccb7c8dc8f6c4a1c9811ac13025539ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:46.769791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.770453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.771972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1183/2355] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:46 2022] DELETE /identity/v3/projects/ccb7c8dc8f6c4a1c9811ac13025539ba => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:46.776033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:46.788808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285583ea-e74b-483a-a012-455c12452dfb service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:46.855787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:46.856488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:46.856777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:46.857063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] PATH_INFO: `/v3/users/1fb25f24d1a7419c9e54046691f24d7c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:46.858231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb25f24d1a7419c9e54046691f24d7c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:46.860227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:46.916015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb25f24d1a7419c9e54046691f24d7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.926503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb25f24d1a7419c9e54046691f24d7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.927225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb25f24d1a7419c9e54046691f24d7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.928448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb25f24d1a7419c9e54046691f24d7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.942792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-46656b82-c87d-439e-a3c6-2b3223a1c21f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb25f24d1a7419c9e54046691f24d7c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:46.959858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1173/2356] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:46 2022] DELETE /identity/v3/users/1fb25f24d1a7419c9e54046691f24d7c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:47.244492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.247049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:47.265128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1319f1-f7b0-4da4-94e0-728ff0b1a06d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.326866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.327507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.327786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.328056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] PATH_INFO: `/v3/projects/ec15397431774f96a0e5f1d749ccd19a` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.329249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec15397431774f96a0e5f1d749ccd19a)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.331109 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.378675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec15397431774f96a0e5f1d749ccd19a', 'request_id': 'req-a74478b0-e1b8-4299-9b74-f209fba6a896'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:47.385423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec15397431774f96a0e5f1d749ccd19a', 'request_id': 'req-a74478b0-e1b8-4299-9b74-f209fba6a896'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:47.390986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec15397431774f96a0e5f1d749ccd19a', 'request_id': 'req-a74478b0-e1b8-4299-9b74-f209fba6a896'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:47.402103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] The token cache is being invalidate because project ec15397431774f96a0e5f1d749ccd19a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:47.402392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:47.403020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a74478b0-e1b8-4299-9b74-f209fba6a896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:47.404667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1184/2357] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:47 2022] DELETE /identity/v3/projects/ec15397431774f96a0e5f1d749ccd19a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:47.409732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.410065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.410323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.431282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:47.433153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:47.473270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:47.494319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1174/2358] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:47.503486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.503775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.504055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.524737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:47.528589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:47.537211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1185/2359] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:47.541310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.552237 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2275ba5a-cdb6-465e-8d19-6440d5932640 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.556746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.557581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.557950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.558229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.558916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:47.559218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.560844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.572480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1175/2360] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:47.577748 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.589071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af13e2-2b77-487c-be7a-3e6877dd1919 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.593700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.594317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.594596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.594864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.595518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.597435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.630923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1186/2361] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:47 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:47.632681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.643569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c189a-6b42-4712-99a7-465c459583f3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.707744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.708277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.708589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.708862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.709523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.711424 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.739117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1176/2362] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:47 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:47.740300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.751067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263faaa-1690-46f7-8bb0-399236c66ddb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.755628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.756234 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.756622 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.756891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.757578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.759161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.773465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1187/2363] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:47.774366 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.785383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9ea12-d595-4951-acea-65b72442e65a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.790224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.790842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.791122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.791390 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.793195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.794815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25875cde-2275-46b7-a4b3-396c25a5f317 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.806163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1177/2364] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:01:47.813326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:47.821055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1851e912-d08a-42f8-ad87-853a1ce46b37 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:47.826209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:47.827050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.827405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.827750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] PATH_INFO: `/v3/projects/3dfa6aa616e94d46a68abcd3b5a15711/users/16472bad30f3495da19c79176cfdb745/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.857015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfa6aa616e94d46a68abcd3b5a15711, user_id=16472bad30f3495da19c79176cfdb745, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:47.859880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:47.869388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:47.874685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1188/2365] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:47 2022] PUT /identity/v3/projects/3dfa6aa616e94d46a68abcd3b5a15711/users/16472bad30f3495da19c79176cfdb745/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:47.885024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:47.885833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:47.886193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:47.919491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] MFA Rules not processed for user `16472bad30f3495da19c79176cfdb745`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:47.920764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:48.000875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1178/2366] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:01:48.005927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.114207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f40bac-5fb0-4c43-b8b3-6d56db80f2d8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.121089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.121994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.122359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.123146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.123446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.125538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.163592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1189/2367] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:48 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:48.169107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.181052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095b055-5aea-4345-aca3-d6826676df48 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.228634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:48.257016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.257627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.257937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.258232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.258941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.261408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.287340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1179/2368] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:48 2022] POST /identity/v3/users => generated 300 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:48.291579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.303837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5218b-368b-4459-88a9-c79878dde84b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.309685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.310363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.310641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.310920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.311654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.313259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.325951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1190/2369] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:48.330678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.342382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721013c-53b5-4fbc-afb1-c273df55f064 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.347155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.347819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.348102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.348410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] PATH_INFO: `/v3/projects/064161c346df4dafa0788d69ffb340fb/users/e849cd1b80e94d2b86ec5b9959edcbbc/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.365518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064161c346df4dafa0788d69ffb340fb, user_id=e849cd1b80e94d2b86ec5b9959edcbbc, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.368266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.382872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1180/2370] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:48 2022] PUT /identity/v3/projects/064161c346df4dafa0788d69ffb340fb/users/e849cd1b80e94d2b86ec5b9959edcbbc/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:48.387125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.397880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82158e2f-c9c0-4723-b571-ea73abe5a7c1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.463871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.465145 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.465433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.465708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.466409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.468048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f7869-9372-4ce1-8cd2-b3d2aecca153 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.480293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1191/2371] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:01:48.485370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.498873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461cb0-8e3f-4f77-abd6-986a6cf99b62 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.503996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.504714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.504997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.505307 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] PATH_INFO: `/v3/projects/064161c346df4dafa0788d69ffb340fb/users/e849cd1b80e94d2b86ec5b9959edcbbc/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.507289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064161c346df4dafa0788d69ffb340fb, user_id=e849cd1b80e94d2b86ec5b9959edcbbc, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.509938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.527479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1181/2372] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:48 2022] PUT /identity/v3/projects/064161c346df4dafa0788d69ffb340fb/users/e849cd1b80e94d2b86ec5b9959edcbbc/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:48.539481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.539764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.540088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.574005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] MFA Rules not processed for user `e849cd1b80e94d2b86ec5b9959edcbbc`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:48.575802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:48.649140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:48.664934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1192/2373] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:48.677814 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.680098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:48.745829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317417fc-360c-455b-88e2-8448fd796386 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.749712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.750347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.750622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.751603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.765236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.766898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9299661f-fb22-4b2b-a199-42a97fdd1028 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.778224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1182/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:48.813785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:48.881736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81752b-1dcf-443a-b71a-045d63c1ba6d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:48.886994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:48.887645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:48.887932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:48.888201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] PATH_INFO: `/v3/users/16472bad30f3495da19c79176cfdb745` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:48.889417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=16472bad30f3495da19c79176cfdb745)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:48.891344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:48.952566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16472bad30f3495da19c79176cfdb745'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:48.953251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16472bad30f3495da19c79176cfdb745'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:48.965184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16472bad30f3495da19c79176cfdb745'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:48.973528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16472bad30f3495da19c79176cfdb745'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:48.974211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16472bad30f3495da19c79176cfdb745'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.000614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1193/2375] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:48 2022] DELETE /identity/v3/users/16472bad30f3495da19c79176cfdb745 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:49.005442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:49.005944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:49.006044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:49.025525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:49.027893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:49.104985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1183/2376] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:01:49.117593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:49.119371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:49.194517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb305157-7d2a-4c6e-a777-29bd6603f84d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:49.198445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:49.199057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:49.199350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:49.199627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:49.210484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:49.212253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:49.223698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1194/2377] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:49.597509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:49.599380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:49.613269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740deb32-3101-4d62-a73d-7bafd795bd3c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:49.617947 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:49.618612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:49.618900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:49.619782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] PATH_INFO: `/v3/projects/3dfa6aa616e94d46a68abcd3b5a15711` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:49.620900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfa6aa616e94d46a68abcd3b5a15711)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:49.622859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:49.666079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:49.691780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfa6aa616e94d46a68abcd3b5a15711', 'request_id': 'req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.700719 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfa6aa616e94d46a68abcd3b5a15711', 'request_id': 'req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.711388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfa6aa616e94d46a68abcd3b5a15711', 'request_id': 'req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.726090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] The token cache is being invalidate because project 3dfa6aa616e94d46a68abcd3b5a15711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:49.726378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.727167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.730127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1184/2378] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:49 2022] DELETE /identity/v3/projects/3dfa6aa616e94d46a68abcd3b5a15711 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:49.734340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:49.747088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00781289-14a9-4e0a-8053-9b927315ad68 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:49.761528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:49.812190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:49.812898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:49.813182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:49.813460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] PATH_INFO: `/v3/users/e849cd1b80e94d2b86ec5b9959edcbbc` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:49.814623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e849cd1b80e94d2b86ec5b9959edcbbc)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:49.816585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:49.866071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e849cd1b80e94d2b86ec5b9959edcbbc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.866644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e849cd1b80e94d2b86ec5b9959edcbbc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.874399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e849cd1b80e94d2b86ec5b9959edcbbc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.881656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e849cd1b80e94d2b86ec5b9959edcbbc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.882307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-486d4b26-0c1a-4538-ad7d-eac7b55bbd73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e849cd1b80e94d2b86ec5b9959edcbbc'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:49.905576 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1195/2379] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:49 2022] DELETE /identity/v3/users/e849cd1b80e94d2b86ec5b9959edcbbc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:50.225310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.226842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:50.241681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c838ae4-3c76-42ec-a2c7-6fef99d96afd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.303595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.304299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.304685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.304882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] PATH_INFO: `/v3/projects/064161c346df4dafa0788d69ffb340fb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.306065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] RBAC: Authorizing `identity:delete_project(project_id=064161c346df4dafa0788d69ffb340fb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.308068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.357820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064161c346df4dafa0788d69ffb340fb', 'request_id': 'req-1fc9a063-0a39-4445-8f6b-d3adffe2252b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:50.365486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064161c346df4dafa0788d69ffb340fb', 'request_id': 'req-1fc9a063-0a39-4445-8f6b-d3adffe2252b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:50.373982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064161c346df4dafa0788d69ffb340fb', 'request_id': 'req-1fc9a063-0a39-4445-8f6b-d3adffe2252b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:50.385592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] The token cache is being invalidate because project 064161c346df4dafa0788d69ffb340fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:50.386442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:50.387098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1fc9a063-0a39-4445-8f6b-d3adffe2252b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:50.388737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1185/2380] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:50 2022] DELETE /identity/v3/projects/064161c346df4dafa0788d69ffb340fb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:50.393745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.394031 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.394307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.418001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:50.419232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:50.489213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1196/2381] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:50.497958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.498447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.498738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.531498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:50.532736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:50.541695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1186/2382] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:50.549735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.564867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e2f10-e310-42e7-b95d-a50b1ab1d72f None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.569488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.570299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.570588 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.570857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.571744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:50.572048 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.574969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.587790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1197/2383] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:01:50.592762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.603284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9b32d-e9aa-45c2-b2fd-2c513a10976b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.608644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.609574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.609886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.610160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.610816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.612790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.645054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1187/2384] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:50 2022] POST /identity/v3/projects => generated 361 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:50.648058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.664239 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6ca2c-f581-456c-acfb-6dffe54c1dce admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.687150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:50.735135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.735762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.736041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.736312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.737174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.739080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.767620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1198/2385] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:50 2022] POST /identity/v3/users => generated 281 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:50.768898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.785582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e07885-0c72-40c3-9699-70d0e034a69f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.793255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.795028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.795028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.795028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.795308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.796943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.814620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1188/2386] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:50.818865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.829514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec049cf-99af-4dee-b5f5-e91f080f0ec7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.835132 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.835751 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.836030 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.836307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.837061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.839169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9585dd-e32e-4118-9b7b-55bd7672bb0c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.851610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1199/2387] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:50.857345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:50.872147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85cb4e6-4ee0-428c-941c-d6d8941d5700 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:50.878277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:50.878987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.879275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.879557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] PATH_INFO: `/v3/projects/8e3264d0b1f54c93ab52dd2036996e29/users/7de60b4fec0f4b9c8c99c10906cd02f6/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.883091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:50.898984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3264d0b1f54c93ab52dd2036996e29, user_id=7de60b4fec0f4b9c8c99c10906cd02f6, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:50.901692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:50.915537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1189/2388] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:50 2022] PUT /identity/v3/projects/8e3264d0b1f54c93ab52dd2036996e29/users/7de60b4fec0f4b9c8c99c10906cd02f6/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:50.927103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:50.927380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:50.927659 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:50.959621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] MFA Rules not processed for user `7de60b4fec0f4b9c8c99c10906cd02f6`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:50.960872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:51.031618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1200/2389] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:51.070502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:51.072584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:51.135801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87be72f-1464-4fff-a974-309a90e100e1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:51.139969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:51.140635 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.140919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.141199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:51.150706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:51.152109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0cc2c-b172-4a79-9c13-3672affd7c1a service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:51.161165 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1190/2390] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:51.176915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:51.248606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159a035-bbfb-4196-a7df-1c98afdbc487 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:51.253190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:51.253835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.254115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.254396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] PATH_INFO: `/v3/users/7de60b4fec0f4b9c8c99c10906cd02f6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:51.255725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de60b4fec0f4b9c8c99c10906cd02f6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:51.257986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:51.296664 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:51.309458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de60b4fec0f4b9c8c99c10906cd02f6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.310119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de60b4fec0f4b9c8c99c10906cd02f6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.317139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de60b4fec0f4b9c8c99c10906cd02f6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.324035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de60b4fec0f4b9c8c99c10906cd02f6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.324764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de60b4fec0f4b9c8c99c10906cd02f6'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.342880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1201/2391] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:01:51 2022] DELETE /identity/v3/users/7de60b4fec0f4b9c8c99c10906cd02f6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:51.347380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.347880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.347992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:51.367462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:51.368705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:51.432260 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1191/2392] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:51.442321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:51.443921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:51.501179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ce2d-9b27-4d9f-967a-658c3af60858 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:51.504833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:51.505703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.505703 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.505999 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:51.515124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:51.516726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeea0a7-0331-464c-bbd5-1e79b2718a32 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:51.527838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1202/2393] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:51.884891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:51.887158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:51.901755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8060975-0f00-4f55-8fa8-6687f2a31a9f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:51.907167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:51.907769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.908056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.908362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] PATH_INFO: `/v3/projects/8e3264d0b1f54c93ab52dd2036996e29` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:51.909590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3264d0b1f54c93ab52dd2036996e29)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:51.911477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:51.959902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3264d0b1f54c93ab52dd2036996e29', 'request_id': 'req-960db0c4-4e24-4e40-991f-7be581ea8ad4'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.968468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3264d0b1f54c93ab52dd2036996e29', 'request_id': 'req-960db0c4-4e24-4e40-991f-7be581ea8ad4'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.974039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3264d0b1f54c93ab52dd2036996e29', 'request_id': 'req-960db0c4-4e24-4e40-991f-7be581ea8ad4'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.985230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] The token cache is being invalidate because project 8e3264d0b1f54c93ab52dd2036996e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:01:51.985517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.986267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-960db0c4-4e24-4e40-991f-7be581ea8ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:01:51.987902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1192/2394] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:01:51 2022] DELETE /identity/v3/projects/8e3264d0b1f54c93ab52dd2036996e29 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:51.992873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:51.993372 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:51.993489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.013792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:52.014996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:52.074175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1203/2395] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:52.083054 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.083342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.083619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.103668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:52.105386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:52.113289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1193/2396] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:52.117611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.127923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a5d7e-e961-4daf-95bb-73719e0afcc3 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.132532 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.133138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.133418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.133688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.134369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:01:52.134672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.137011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.150283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1204/2397] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:01:52.156568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.162392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:52.166854 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31962b-8098-4ac0-8dba-dea0af424d39 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.172797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.173439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.173915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.174248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.175304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.177452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.205537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1194/2398] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:52.209950 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.219928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5065be40-434b-45fb-890e-5dea8f549d64 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.261548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:52.281086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.281930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.282236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.282508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.283295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.285415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.309847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1205/2399] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:52.313991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.325427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1721657-3c25-4818-81ad-c67647cb2465 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.330364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.331002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.331305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.331623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.332477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.334130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.344675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1195/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:52.348615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.359499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32f061-fff1-4412-96d1-8c93b98c259d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.364103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.364902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.365179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.365450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.366144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.367726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872f3c-c659-4fca-8ddc-92d2d7e04f3d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.380204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1206/2401] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:52.384457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.395481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00902034-9bc0-4deb-9256-395f0738c1c4 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.400085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.401004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.401294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.401580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] PATH_INFO: `/v3/projects/77a694c506514d09b88e0b4290d27a26/users/1a0a2cc210a745f090d189b371549919/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.418379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77a694c506514d09b88e0b4290d27a26, user_id=1a0a2cc210a745f090d189b371549919, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.420999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.434896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1196/2402] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:52 2022] PUT /identity/v3/projects/77a694c506514d09b88e0b4290d27a26/users/1a0a2cc210a745f090d189b371549919/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:52.445042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.445332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.445605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.475093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] MFA Rules not processed for user `1a0a2cc210a745f090d189b371549919`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:52.476299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:52.544096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1207/2403] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:52.549285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.612394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358dd07-1e4f-42fc-9ed8-b3d919e27589 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.617039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.617650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.617932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.618206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.618874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.621653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.648136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1197/2404] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:52.652367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.662771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705f1ff3-47af-4e58-8188-5134145b7e86 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.721316 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.721846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.722128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.722401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.723067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.725034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.746917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1208/2405] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:52.750969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.761486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108a520-75cd-4c66-bf10-1959b1afcfe5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.765998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.766605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.766887 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.767191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.767870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.769500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.780892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1198/2406] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:52.784944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.795961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d374f5-6d60-4ab4-865b-5b81e1a9b6d5 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.801039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.801655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.801938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.802250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.803026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.804687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9bbf52-ee04-4dfc-ba41-2e8bc4825f16 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.815837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1209/2407] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:52.820099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:52.830843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b6f9a-ee09-4294-b15a-8842ec750fa5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:52.835924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:52.836592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.836878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.837173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] PATH_INFO: `/v3/projects/056aad14e46846c696368deddea08bf3/users/a0bd971328744fd096696a820853f3ab/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.853121 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056aad14e46846c696368deddea08bf3, user_id=a0bd971328744fd096696a820853f3ab, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:52.856619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:52.869861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1199/2408] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:52 2022] PUT /identity/v3/projects/056aad14e46846c696368deddea08bf3/users/a0bd971328744fd096696a820853f3ab/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:52.879943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:52.880262 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:52.880670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:52.911555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] MFA Rules not processed for user `a0bd971328744fd096696a820853f3ab`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:52.912936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:52.931098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:52.996182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1210/2409] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:52.997695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.076537 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:53.080050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b7c39-93ca-41c0-9e7a-8ffc65568609 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.085523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.086161 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.086441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.086736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.087395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.090143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.117409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1200/2410] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:01:52 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:01:53.121868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.132793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c069ded4-f842-4d85-bbef-69e45526ea07 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.198620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.199249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.199535 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.199812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.200538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.202527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.225093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1211/2411] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:01:53 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:01:53.230000 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.241035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002a9650-b50a-4edc-bb22-38bbdfdb165c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.250115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.251026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.251454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.251733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.252484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.254104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.266431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1201/2412] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:01:53.270778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.281133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318e9ab-9597-4ce9-8ff1-f81604012b8e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.285747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.286397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.286678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.286954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] PATH_INFO: `/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7/users/f75db0bf5d3d4e2a8f45071bb65bfb22/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.302920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003c3b0b1eae40fab5c2e132b2b437f7, user_id=f75db0bf5d3d4e2a8f45071bb65bfb22, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.306341 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.321072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1212/2413] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:53 2022] PUT /identity/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7/users/f75db0bf5d3d4e2a8f45071bb65bfb22/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:01:53.325365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.335855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb1df0-8294-40cf-9029-615ac6b67223 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.398281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.399022 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.399168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.399445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.400134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.401905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adcf0a-1e99-44fe-86d2-14cd15e040ec admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.414522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1202/2414] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:01:53.419867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.430357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864808ac-be40-40d8-b2a6-8feb14cc9250 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.436023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.436712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.436993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.437270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] PATH_INFO: `/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7/users/f75db0bf5d3d4e2a8f45071bb65bfb22/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.439174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003c3b0b1eae40fab5c2e132b2b437f7, user_id=f75db0bf5d3d4e2a8f45071bb65bfb22, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.441851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.452604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1213/2415] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:01:53 2022] PUT /identity/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7/users/f75db0bf5d3d4e2a8f45071bb65bfb22/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:01:53.463611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89606436-985b-49e8-a521-6fa590d1645f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.464002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89606436-985b-49e8-a521-6fa590d1645f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.464382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89606436-985b-49e8-a521-6fa590d1645f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.492417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-89606436-985b-49e8-a521-6fa590d1645f None None] MFA Rules not processed for user `f75db0bf5d3d4e2a8f45071bb65bfb22`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:01:53.493683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-89606436-985b-49e8-a521-6fa590d1645f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:53.563584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1203/2416] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:01:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:01:53.575141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:53.576969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:01:53.634870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a55a11-f8d2-4fca-a116-93ceb004add1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:53.638593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:53.639468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:53.639758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:53.640032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:53.665585 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:53.700524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:53.701957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:53.710937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1214/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:55.215601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89606436-985b-49e8-a521-6fa590d1645f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:55.225523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89606436-985b-49e8-a521-6fa590d1645f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:55.229381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:55.230115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:55.230407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:55.230686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:55.281798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:01:55.293202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:55.294907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:55.305583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1204/2418] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:01:55.946276 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:01:55.956789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572561-ba1f-41b7-afbc-c769ed43ba7e service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:01:55.960781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:01:55.962130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:01:55.962130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:01:55.962130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:01:55.973282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:01:55.975212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77f55a-7e4e-4235-9357-6234c34cf895 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:01:55.987333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1215/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:01:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:00.927680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:00.990764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8176fd-52eb-4682-b484-bd512b4bdc12 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:00.995873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:00.996547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:00.996834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:00.997110 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] PATH_INFO: `/v3/users/1a0a2cc210a745f090d189b371549919` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:00.998274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a0a2cc210a745f090d189b371549919)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:01.000180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:01.047819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a0a2cc210a745f090d189b371549919'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.055388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a0a2cc210a745f090d189b371549919'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.056005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a0a2cc210a745f090d189b371549919'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.056669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a0a2cc210a745f090d189b371549919'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.068586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a0a2cc210a745f090d189b371549919'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.080844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1205/2420] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:02:00 2022] DELETE /identity/v3/users/1a0a2cc210a745f090d189b371549919 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:01.085383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:01.085900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:01.086008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:01.105655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:01.106762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:01.172089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1216/2421] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:01.183227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:01.184632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:01.242971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ff544-afb9-481b-8c12-5cee87686bc6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:01.247230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:01.247848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:01.248133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:01.248471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:01.257668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:01.259095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:01.269008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1206/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:01.544729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:01.546931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:01.563537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ce3320-5106-466d-ba77-1603d273df11 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:01.568794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:01.569608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:01.569895 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:01.570174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] PATH_INFO: `/v3/projects/77a694c506514d09b88e0b4290d27a26` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:01.571396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77a694c506514d09b88e0b4290d27a26)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:01.573838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:01.623050 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77a694c506514d09b88e0b4290d27a26', 'request_id': 'req-00f753b0-d3b2-4134-9d55-59cc1ce659d8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.630492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77a694c506514d09b88e0b4290d27a26', 'request_id': 'req-00f753b0-d3b2-4134-9d55-59cc1ce659d8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.637055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77a694c506514d09b88e0b4290d27a26', 'request_id': 'req-00f753b0-d3b2-4134-9d55-59cc1ce659d8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.648518 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] The token cache is being invalidate because project 77a694c506514d09b88e0b4290d27a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:02:01.648805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.649432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.652178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1217/2423] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:02:01 2022] DELETE /identity/v3/projects/77a694c506514d09b88e0b4290d27a26 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:02:01.656112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:01.665568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c2e4d-1c26-4790-b4a4-e7ef45420a7d service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:01.674782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:01.724418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:01.725060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:01.725342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:01.725619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] PATH_INFO: `/v3/users/a0bd971328744fd096696a820853f3ab` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:01.726765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bd971328744fd096696a820853f3ab)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:01.728805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:01.782684 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bd971328744fd096696a820853f3ab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.790279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bd971328744fd096696a820853f3ab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.791243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bd971328744fd096696a820853f3ab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.792092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bd971328744fd096696a820853f3ab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.804220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bd971328744fd096696a820853f3ab'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:01.817529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1207/2424] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:02:01 2022] DELETE /identity/v3/users/a0bd971328744fd096696a820853f3ab => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:02.104802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:02.104802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:02.121065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f753b0-d3b2-4134-9d55-59cc1ce659d8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:02.159257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:02.186150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:02.186813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:02.187095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:02.187374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] PATH_INFO: `/v3/projects/056aad14e46846c696368deddea08bf3` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:02.188676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056aad14e46846c696368deddea08bf3)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:02.190589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:02.243683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056aad14e46846c696368deddea08bf3', 'request_id': 'req-90c49a5c-93d6-4748-8373-181407b42ac1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.254129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056aad14e46846c696368deddea08bf3', 'request_id': 'req-90c49a5c-93d6-4748-8373-181407b42ac1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.260978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056aad14e46846c696368deddea08bf3', 'request_id': 'req-90c49a5c-93d6-4748-8373-181407b42ac1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.272682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] The token cache is being invalidate because project 056aad14e46846c696368deddea08bf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:02:02.272962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.273669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.275229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1218/2425] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:02:02 2022] DELETE /identity/v3/projects/056aad14e46846c696368deddea08bf3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:02.280037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:02.281543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:02.296425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be67df-4a8d-44b8-b665-22dd182f2478 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:02.356242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:02.356919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:02.357201 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:02.357481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] PATH_INFO: `/v3/users/f75db0bf5d3d4e2a8f45071bb65bfb22` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:02.358633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75db0bf5d3d4e2a8f45071bb65bfb22)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:02.360609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:02.401613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:02.412281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75db0bf5d3d4e2a8f45071bb65bfb22'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.419620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75db0bf5d3d4e2a8f45071bb65bfb22'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.420261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75db0bf5d3d4e2a8f45071bb65bfb22'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.420919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75db0bf5d3d4e2a8f45071bb65bfb22'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.434402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0fab3731-7706-40f3-92b9-97527bb9b8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75db0bf5d3d4e2a8f45071bb65bfb22'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.446799 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1208/2426] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:02:02 2022] DELETE /identity/v3/users/f75db0bf5d3d4e2a8f45071bb65bfb22 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:02.741376 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:02.742896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:02.761373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c49a5c-93d6-4748-8373-181407b42ac1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:02.820991 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:02.821601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:02.821888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:02.822231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] PATH_INFO: `/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:02.824172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] RBAC: Authorizing `identity:delete_project(project_id=003c3b0b1eae40fab5c2e132b2b437f7)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:02.826146 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:02.892810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003c3b0b1eae40fab5c2e132b2b437f7', 'request_id': 'req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.901338 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003c3b0b1eae40fab5c2e132b2b437f7', 'request_id': 'req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.907110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003c3b0b1eae40fab5c2e132b2b437f7', 'request_id': 'req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.909344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:02.921392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] The token cache is being invalidate because project 003c3b0b1eae40fab5c2e132b2b437f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:02:02.921869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.923041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-77e79d6a-ac8d-42a1-a126-6d4abb96c82a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:02.925252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1219/2427] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:02:02 2022] DELETE /identity/v3/projects/003c3b0b1eae40fab5c2e132b2b437f7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:02.929848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:02.930053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f8ed8-7b5c-44cb-ac6c-58a199964e97 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:02.930172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:02.930398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:02.930506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f8ed8-7b5c-44cb-ac6c-58a199964e97 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:02.930835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f8ed8-7b5c-44cb-ac6c-58a199964e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:02.952001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e85f8ed8-7b5c-44cb-ac6c-58a199964e97 None None] MFA Rules not processed for user `3f4c9c7f2ddd4403b13696794dfaab5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:02.953250 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e85f8ed8-7b5c-44cb-ac6c-58a199964e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:02.959628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:02.960881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.008831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1220/2428] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:02:03.025515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1209/2429] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:03.034024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.034665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.034665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.056280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:03.058166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.066326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1221/2430] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:02:03.070955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.083987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f675a-ce73-4d2e-80e4-a6705f3dfe4e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.089162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.089821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.090104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.090441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.091147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:03.091453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.093125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.105801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1210/2431] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:03.110104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.119795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cf2dbe-3b2d-4783-a86e-dcb14ad378c6 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.124844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.125470 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.125754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.126029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.126711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.128635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.159951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1222/2432] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:03.161429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.172609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d0a58-84eb-4e0d-8828-01ba8a61a1ed admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.231464 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.232093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.232404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.232687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.233365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.235285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.257290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1211/2433] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:03.261951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.273268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48caa73-e76d-449c-88af-4ffe15e10af3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.278208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.278846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.279123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.279389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.280090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.281708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.292076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1223/2434] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:02:03.296102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.308006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc272a0-895d-456b-960c-f293d671eabe admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.312736 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.313361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.313642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.313918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.314621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.316410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a1b70-b216-426a-b944-96afa272c3b9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.328722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1212/2435] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:02:03.333067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.343932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5728b07b-3c86-4401-a265-bfb4cf45fc08 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.348678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.349344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.349620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.349889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] PATH_INFO: `/v3/projects/6b83d22338fe4de581e75d0277105d8c/users/f1643041ce544b60b5ecbc663611b616/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.364830 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b83d22338fe4de581e75d0277105d8c, user_id=f1643041ce544b60b5ecbc663611b616, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.367465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.381668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1224/2436] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:02:03 2022] PUT /identity/v3/projects/6b83d22338fe4de581e75d0277105d8c/users/f1643041ce544b60b5ecbc663611b616/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:03.454490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.454784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.455064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.482724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] MFA Rules not processed for user `f1643041ce544b60b5ecbc663611b616`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:03.483966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.488984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:03.556900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1213/2437] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:03.569287 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:03.571046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:03.630733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ce27e-d374-4837-8ae8-b1716dc5a451 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:03.634383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:03.635655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:03.635932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:03.636209 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:03.645574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:03.647006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58220582-6953-4e6d-94b1-3eb91aeab885 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:03.657208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1225/2438] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:06.262169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:06.354154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c1348-1da8-4884-854c-4b97b117177c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:06.358650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:06.359305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:06.359587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:06.359862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] PATH_INFO: `/v3/users/f1643041ce544b60b5ecbc663611b616` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:06.361058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1643041ce544b60b5ecbc663611b616)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:06.363009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:06.410875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1643041ce544b60b5ecbc663611b616'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:06.419493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1643041ce544b60b5ecbc663611b616'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:06.420198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1643041ce544b60b5ecbc663611b616'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:06.420870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1643041ce544b60b5ecbc663611b616'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:06.433557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1643041ce544b60b5ecbc663611b616'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:06.446146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1214/2439] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:02:06 2022] DELETE /identity/v3/users/f1643041ce544b60b5ecbc663611b616 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:06.451168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:06.451654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:06.451750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:06.471750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:06.473005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:06.507174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:06.535297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1226/2440] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:06.548283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:06.550103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:06.581723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:06.611411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28cdc3-475c-4e91-828e-cac46fe55915 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:06.615067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:06.615698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:06.615980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:06.616258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:06.625890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:06.627323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af96d10c-8317-40fe-b789-50fdbf385a11 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:06.637601 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1215/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:07.002301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.003747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.017654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953eae9-e2e2-48eb-ae75-d7398e494daf None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.022529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.024135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.024451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.024765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] PATH_INFO: `/v3/projects/6b83d22338fe4de581e75d0277105d8c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.026845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b83d22338fe4de581e75d0277105d8c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.028789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.076813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b83d22338fe4de581e75d0277105d8c', 'request_id': 'req-b8fafd36-4c12-4d70-a546-2309f52e7a6e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:07.084176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b83d22338fe4de581e75d0277105d8c', 'request_id': 'req-b8fafd36-4c12-4d70-a546-2309f52e7a6e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:07.091081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b83d22338fe4de581e75d0277105d8c', 'request_id': 'req-b8fafd36-4c12-4d70-a546-2309f52e7a6e'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:07.102084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] The token cache is being invalidate because project 6b83d22338fe4de581e75d0277105d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:02:07.102371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:07.103007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b8fafd36-4c12-4d70-a546-2309f52e7a6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:07.105512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1227/2442] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:02:07 2022] DELETE /identity/v3/projects/6b83d22338fe4de581e75d0277105d8c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:07.110698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.110984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.111261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.130930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:07.132147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.191175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1216/2443] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:07.198711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.199000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.199305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.220056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:07.221371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.230214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1228/2444] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:07.234602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.244258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd1dd3-d7fa-4a14-ae25-75c81f0daae8 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.248884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.249491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.249767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.250043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.250724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:02:07.251016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.252818 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.268308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1217/2445] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:02:07.270210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.280792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f3d1-0c9d-4e70-a23f-7e6f0259ba70 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.285496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.286128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.286534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.286929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.287632 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.289613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.320374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1229/2446] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:07.324038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.334883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784eb92b-14ee-48a9-a086-b4aaaa9edd5b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.395298 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.395922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.396207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.396522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.397249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.399160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.421478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1218/2447] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:02:07.425873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.436858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45cca3-f12a-4b2d-b342-1bbe42ff2bd6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.441462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.442069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.442357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.442627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.443310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.444972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.455838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1230/2448] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:02:07.459828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.473021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041922e3-6e58-49c5-ba11-ab53aa34c9d8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.477911 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.478744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.479039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.479312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.480040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.481688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff73b2-631d-4faa-ad0a-113670d88077 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.493471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1219/2449] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:02:07.497684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.507591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f95ac-ba0b-4105-9c11-4e1ee4a862af admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.512774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.513434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.513722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.514005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] PATH_INFO: `/v3/projects/910c69679a094c3fba35a182c991accc/users/881fce4ba56a4454970886a4cc3b1dad/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.529552 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=910c69679a094c3fba35a182c991accc, user_id=881fce4ba56a4454970886a4cc3b1dad, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.532277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.546095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1231/2450] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:02:07 2022] PUT /identity/v3/projects/910c69679a094c3fba35a182c991accc/users/881fce4ba56a4454970886a4cc3b1dad/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:07.556746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5bb6f-1e13-406b-837b-cc8b2797b71b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.557037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5bb6f-1e13-406b-837b-cc8b2797b71b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.557336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5bb6f-1e13-406b-837b-cc8b2797b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.584619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-96d5bb6f-1e13-406b-837b-cc8b2797b71b None None] MFA Rules not processed for user `881fce4ba56a4454970886a4cc3b1dad`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:07.585836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-96d5bb6f-1e13-406b-837b-cc8b2797b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.609790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:07.651935 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1220/2451] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:07.662615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:07.664286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:07.724657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdca52-2e2b-40fe-8e2d-e0142c116aa1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:07.728427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:07.729040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:07.729320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:07.729590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:07.737978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:07.739503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:07.749153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1232/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:09.125952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bdc8a-d6e6-432e-9c99-6f9e032873b0 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:09.126467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bdc8a-d6e6-432e-9c99-6f9e032873b0 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:09.126562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bdc8a-d6e6-432e-9c99-6f9e032873b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:09.146001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-108bdc8a-d6e6-432e-9c99-6f9e032873b0 None None] MFA Rules not processed for user `881fce4ba56a4454970886a4cc3b1dad`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:09.147221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-108bdc8a-d6e6-432e-9c99-6f9e032873b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:09.155078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1221/2453] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:17.744994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:19.140097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=19 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:20.529066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:20.530466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:20.587665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e9e40-3ccc-4bd0-b168-295f2c3200cc service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:20.591490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:20.592101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:20.592401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:20.592671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:20.594609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:20.657683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:20.659119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:20.669831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1233/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:27.607428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe354be-2003-4dcd-bb4c-68059ae8eb17 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.607974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe354be-2003-4dcd-bb4c-68059ae8eb17 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.607974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe354be-2003-4dcd-bb4c-68059ae8eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.651959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2fe354be-2003-4dcd-bb4c-68059ae8eb17 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:27.653430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2fe354be-2003-4dcd-bb4c-68059ae8eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:27.736159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1222/2455] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 21:02:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:27.782248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:27.783631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:27.893989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6679d97e-fbae-45df-8d9f-ce3c74cc3ebf service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:27.898937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:27.900091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:27.900405 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:27.900688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:27.909435 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:27.911055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:27.916822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:27.920961 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1234/2456] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:02:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:02:28.429711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b34ac0-ec19-438e-893c-01c2440b457a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.430004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b34ac0-ec19-438e-893c-01c2440b457a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.430293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b34ac0-ec19-438e-893c-01c2440b457a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.450513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-52b34ac0-ec19-438e-893c-01c2440b457a None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:28.451722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-52b34ac0-ec19-438e-893c-01c2440b457a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:28.458852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1223/2457] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:28.472125 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.483805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62041ae7-6d88-4c94-bb04-6aa6f2d60d4b service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.489786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.490474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.490760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.491119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.502380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.503972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.514432 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1235/2458] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:02:28.763377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5db7-5b0b-42c3-9283-c3892685554a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.763937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5db7-5b0b-42c3-9283-c3892685554a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.763937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5db7-5b0b-42c3-9283-c3892685554a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.787437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-65fe5db7-5b0b-42c3-9283-c3892685554a None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:28.788725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-65fe5db7-5b0b-42c3-9283-c3892685554a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:28.797042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1224/2459] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 21:02:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:28.810561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:28.822218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc64de-66f5-481b-b68d-7ad706fc38dc service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:28.827629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:28.828246 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:28.828562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:28.828839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:28.838699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:28.840255 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:28.854329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1236/2460] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:02:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:02:29.221125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fec570-c716-494c-ad26-8822901099b9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.221425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fec570-c716-494c-ad26-8822901099b9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.221723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fec570-c716-494c-ad26-8822901099b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.244489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-45fec570-c716-494c-ad26-8822901099b9 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:29.245737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-45fec570-c716-494c-ad26-8822901099b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:29.254513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1225/2461] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:29.271328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:29.292677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324d440-ad01-4654-b920-5aea458c191c service swift] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:29.297951 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:29.298568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:29.298851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:29.299129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:29.309827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:29.311677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f50a2-c1be-43fc-876e-72f44c737246 service swift] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:29.320857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1237/2462] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:02:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:02:31.543619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fec570-c716-494c-ad26-8822901099b9 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.627107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fec570-c716-494c-ad26-8822901099b9 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.631648 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.632380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.632692 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.632987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] PATH_INFO: `/v3/users/881fce4ba56a4454970886a4cc3b1dad` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.634134 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=881fce4ba56a4454970886a4cc3b1dad)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:31.636067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:31.666498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:31.686988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '881fce4ba56a4454970886a4cc3b1dad'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:31.694516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '881fce4ba56a4454970886a4cc3b1dad'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:31.695111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '881fce4ba56a4454970886a4cc3b1dad'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:31.695701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '881fce4ba56a4454970886a4cc3b1dad'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:31.707378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '881fce4ba56a4454970886a4cc3b1dad'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:31.719567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1226/2463] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:02:31 2022] DELETE /identity/v3/users/881fce4ba56a4454970886a4cc3b1dad => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:02:31.724059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.724610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.724733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.771883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:31.773080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:31.834493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1238/2464] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:02:31.845167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:31.846657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:31.908211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eddbc4-8a6a-4442-be3d-0efa8b1c6979 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:31.912024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:31.912689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:31.912977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:31.913257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:31.924057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:31.926695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:31.938331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1227/2465] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:32.131399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:32.132967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:32.146293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6a1ab-9102-4dd9-b384-b41048962692 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:32.152923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:32.153941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:32.153941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:32.161888 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] PATH_INFO: `/v3/projects/910c69679a094c3fba35a182c991accc` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:32.163208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] RBAC: Authorizing `identity:delete_project(project_id=910c69679a094c3fba35a182c991accc)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:32.169867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:32.222958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '910c69679a094c3fba35a182c991accc', 'request_id': 'req-301074a7-9855-4f04-b23c-a5cfc3077831'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:32.231714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '910c69679a094c3fba35a182c991accc', 'request_id': 'req-301074a7-9855-4f04-b23c-a5cfc3077831'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:32.240135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '910c69679a094c3fba35a182c991accc', 'request_id': 'req-301074a7-9855-4f04-b23c-a5cfc3077831'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:32.252062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] The token cache is being invalidate because project 910c69679a094c3fba35a182c991accc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:02:32.252362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:32.253011 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-301074a7-9855-4f04-b23c-a5cfc3077831 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:02:32.254502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1239/2466] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:02:32 2022] DELETE /identity/v3/projects/910c69679a094c3fba35a182c991accc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:02:41.933780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=6,keystone:SELECT=45,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:42.249007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:02:55.670254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:02:55.672397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:55.787771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a5a1cd-2fd4-4f1e-8318-c0b1a3d6a694 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:02:55.851243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:02:55.852422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:55.852713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:55.852990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:55.854866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:55.867218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:02:55.868903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:02:55.879740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1228/2467] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:02:58.659366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:02:58.659920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:02:58.659920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:02:58.680499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] MFA Rules not processed for user `807855d862b146bdba009626f28d8d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:02:58.681709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:02:58.737003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1240/2468] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:03:05.875179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=67 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:06.918179 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:06.919815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:07.011211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8577e09-5090-4afc-9de3-3002fd04d4f3 service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:07.015393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:07.016153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:07.016471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:07.016754 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:07.018796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:07.107355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:07.108852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:07.119044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1229/2469] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:03:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:03:08.730352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=25 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:14.270301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:14.281526 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1481beb-a9b6-4315-b1bd-c3389dea57e8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:14.285491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:14.286247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:14.286527 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:14.286801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:14.288799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:14.408667 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:14.410476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:14.435731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1241/2470] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:03:17.114274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=57 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:17.937604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:17.939097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:17.958133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e801a04-0abc-4f1c-b043-38e527029cd0 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:17.963649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:17.964281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:17.964598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:17.964876 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:17.966953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:17.994264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:17.994264 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:18.005970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1230/2471] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:03:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:03:21.271600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.275063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:21.291202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8a374-026b-488e-95a2-7c129ff733cc service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.295900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.296661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.297000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.297343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] PATH_INFO: `/v3/users/6398ec15831f40bfb1b316070a78adba` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.298586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6398ec15831f40bfb1b316070a78adba)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.301002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.349415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6398ec15831f40bfb1b316070a78adba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.350085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6398ec15831f40bfb1b316070a78adba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.358701 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6398ec15831f40bfb1b316070a78adba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.366130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6398ec15831f40bfb1b316070a78adba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.366901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6398ec15831f40bfb1b316070a78adba'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.384252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1242/2472] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:03:21 2022] DELETE /identity/v3/users/6398ec15831f40bfb1b316070a78adba => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:21.562045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.575133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f8c06-942d-433d-a587-d5619333f2f2 service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.636724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.637372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.637656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.637932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] PATH_INFO: `/v3/projects/a5dee6a2bfb44b53924cf6d76c5fba12` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.639379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5dee6a2bfb44b53924cf6d76c5fba12)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.641038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.687698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5dee6a2bfb44b53924cf6d76c5fba12', 'request_id': 'req-d7310a4f-22cf-40c2-a744-439db6326824'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.695815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5dee6a2bfb44b53924cf6d76c5fba12', 'request_id': 'req-d7310a4f-22cf-40c2-a744-439db6326824'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.702017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5dee6a2bfb44b53924cf6d76c5fba12', 'request_id': 'req-d7310a4f-22cf-40c2-a744-439db6326824'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.713808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] The token cache is being invalidate because project a5dee6a2bfb44b53924cf6d76c5fba12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:03:21.714095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.714738 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.716426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1231/2473] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:03:21 2022] DELETE /identity/v3/projects/a5dee6a2bfb44b53924cf6d76c5fba12 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:21.720422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:21.721822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:21.735107 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d490c-a2dd-4216-94f0-50f7b237cd22 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:21.793758 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:21.794397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:21.794682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:21.794959 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] PATH_INFO: `/v3/users/4236ad72ab0346be8145d528bbba7bbb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:21.796201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4236ad72ab0346be8145d528bbba7bbb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:21.798266 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:21.804283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:21.845831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4236ad72ab0346be8145d528bbba7bbb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.847590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4236ad72ab0346be8145d528bbba7bbb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.855015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4236ad72ab0346be8145d528bbba7bbb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.862580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4236ad72ab0346be8145d528bbba7bbb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.862913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2a34e48-2dbc-482f-8b00-1d9b1e369054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4236ad72ab0346be8145d528bbba7bbb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:21.881292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1243/2474] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:03:21 2022] DELETE /identity/v3/users/4236ad72ab0346be8145d528bbba7bbb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:22.117191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:22.119052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:22.135123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a4f-22cf-40c2-a744-439db6326824 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:22.197843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:22.198499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:22.198783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:22.199060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] PATH_INFO: `/v3/projects/be8305125d0d469788b3dfa44aeb50c8` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:22.200250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8305125d0d469788b3dfa44aeb50c8)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:22.202202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:22.225525 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:22.258244 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8305125d0d469788b3dfa44aeb50c8', 'request_id': 'req-63e88a79-a74c-49f6-acff-ad7c518c6de8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:22.266166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8305125d0d469788b3dfa44aeb50c8', 'request_id': 'req-63e88a79-a74c-49f6-acff-ad7c518c6de8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:22.272401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8305125d0d469788b3dfa44aeb50c8', 'request_id': 'req-63e88a79-a74c-49f6-acff-ad7c518c6de8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:22.284668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] The token cache is being invalidate because project be8305125d0d469788b3dfa44aeb50c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:03:22.284960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:22.286064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:03:22.287731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1232/2475] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:03:22 2022] DELETE /identity/v3/projects/be8305125d0d469788b3dfa44aeb50c8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:03:25.698187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccde6-332f-4785-9bdb-be43ec8227ff None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.698794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccde6-332f-4785-9bdb-be43ec8227ff None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.698794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dccde6-332f-4785-9bdb-be43ec8227ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.720499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b9dccde6-332f-4785-9bdb-be43ec8227ff None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:25.721817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b9dccde6-332f-4785-9bdb-be43ec8227ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:25.785762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1244/2476] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 21:03:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:03:25.805508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:25.808810 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:25.929292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e88a79-a74c-49f6-acff-ad7c518c6de8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:25.936173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:25.937885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:25.938170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:25.938448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:25.948560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:25.950155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:25.966458 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1233/2477] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:03:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:03:35.786075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:35.956528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:44.787238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfeb72-9212-4b37-bd74-4dd3c861ee57 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.787692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfeb72-9212-4b37-bd74-4dd3c861ee57 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.787847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfeb72-9212-4b37-bd74-4dd3c861ee57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.809592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0ccfeb72-9212-4b37-bd74-4dd3c861ee57 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:03:44.810857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0ccfeb72-9212-4b37-bd74-4dd3c861ee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:03:44.818154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1245/2478] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:03:44.831113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:03:44.842068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9e46f-b19e-4cc9-8155-fbe3e857a83b service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:03:44.847159 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:03:44.847760 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:03:44.848043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:03:44.848319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:03:44.857406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:03:44.858989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:03:44.868187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1234/2479] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:03:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:03:54.803162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:03:54.863865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:07.295041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.295498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.295604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.317019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:07.319313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:07.327271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1246/2480] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:07.340863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:07.351988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e612c-a67a-49ee-a81b-a2fd9f163607 service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:07.357073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:07.357679 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:07.357963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:07.358238 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:07.367305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:07.368940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:07.377428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1235/2481] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:04:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 21:04:13.158223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:13.159597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:13.217113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43019dd-fdd1-4b18-96c2-5dd16423913d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:13.220654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:13.221269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:13.221546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:13.221817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:13.223549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:13.289117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:13.290589 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:13.300766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1247/2482] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:16.321927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.323764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:16.341830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845e7ad-6123-4b02-9b24-e4efcf246819 service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:16.348089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:16.349081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.349371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.349762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] PATH_INFO: `/v3/users/87929879b1c74efea0a390a3ce1a4006` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.351033 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87929879b1c74efea0a390a3ce1a4006)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:16.353094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:16.425228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87929879b1c74efea0a390a3ce1a4006'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.434117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87929879b1c74efea0a390a3ce1a4006'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.434893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87929879b1c74efea0a390a3ce1a4006'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.435528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87929879b1c74efea0a390a3ce1a4006'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.460625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-11fa9b99-524a-4fd1-b597-27b0a29ff2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87929879b1c74efea0a390a3ce1a4006'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.481145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1236/2483] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:04:16 2022] DELETE /identity/v3/users/87929879b1c74efea0a390a3ce1a4006 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:16.659848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.670293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124bbec8-8e55-4a9e-b68e-983991304c54 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:16.729827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:16.730479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.730837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.731124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] PATH_INFO: `/v3/projects/e6ea3d73695c4b3dba67b2da3a4df789` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.732275 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6ea3d73695c4b3dba67b2da3a4df789)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:16.734226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:16.785054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:16.789427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6ea3d73695c4b3dba67b2da3a4df789', 'request_id': 'req-25cdbe71-b215-483c-a7be-5b215c5c2dfd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.797648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6ea3d73695c4b3dba67b2da3a4df789', 'request_id': 'req-25cdbe71-b215-483c-a7be-5b215c5c2dfd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.804449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6ea3d73695c4b3dba67b2da3a4df789', 'request_id': 'req-25cdbe71-b215-483c-a7be-5b215c5c2dfd'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.815782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] The token cache is being invalidate because project e6ea3d73695c4b3dba67b2da3a4df789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:04:16.816068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.816755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-25cdbe71-b215-483c-a7be-5b215c5c2dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:16.818821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1248/2484] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:04:16 2022] DELETE /identity/v3/projects/e6ea3d73695c4b3dba67b2da3a4df789 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:16.823746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.824036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.824345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.873231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:16.874351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:16.933052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1237/2485] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:16.940909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.941231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.941510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.961897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:16.963124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:16.970964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1249/2486] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:16.976853 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:16.986682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0252cb09-85c3-4055-b040-132d40263a3d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:16.991796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:16.992437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:16.992720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:16.993010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:16.993680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:16.993982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:16.995582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.007432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1238/2487] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:17.011592 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.020938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611d75c-b4e4-4de6-8633-af82e823ffff None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.025989 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.026591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.026874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.027149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.027799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.029736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.058460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1250/2488] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:04:17 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:17.062668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.072670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce9d64-b852-4c6d-8799-b1945315b26f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.092258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:17.132256 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.132791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.133067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.133340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.133988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.135920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.159146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1239/2489] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:17 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:17.163952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.174484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1b6e2-6890-4131-af33-0ad70daa482b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.179296 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.180177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.180177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.180510 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.181193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.182818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.192753 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1251/2490] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:17.197097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.207024 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41582c9a-7891-4958-aa23-2ca8710306d6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.212535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.213145 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.213479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.213765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.214573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.216128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c484-986d-41fe-b4f8-7313f47a5aa0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.226875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1240/2491] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:17.231141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.241222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8298b95-da48-4af7-bbbb-ddba244fa73f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.245726 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.246377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.246656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.246927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] PATH_INFO: `/v3/projects/39a105e1d7c54c609b70064dcbfe04a4/users/982fc634537e4ef287966fba2493ffb3/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.262709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] RBAC: Authorizing `identity:create_grant(project_id=39a105e1d7c54c609b70064dcbfe04a4, user_id=982fc634537e4ef287966fba2493ffb3, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.265379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.283219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1252/2492] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:04:17 2022] PUT /identity/v3/projects/39a105e1d7c54c609b70064dcbfe04a4/users/982fc634537e4ef287966fba2493ffb3/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:17.296010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.296379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.296603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.326681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] MFA Rules not processed for user `982fc634537e4ef287966fba2493ffb3`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:17.327942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:17.394035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1241/2493] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:17.406279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:17.407953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:17.515929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365c52b-66b4-4615-8278-2a80b886435f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:17.524281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:17.525152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:17.525575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:17.525975 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:17.537478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:17.539626 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bade5-6422-471b-a8f7-4d86ab9b1e6e service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:17.552156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1253/2494] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:04:20.309829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:20.376410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905dfc4-ce28-4700-8453-ea1eba5dd489 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:20.381345 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:20.382074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.382361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.382641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] PATH_INFO: `/v3/users/982fc634537e4ef287966fba2493ffb3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.383838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=982fc634537e4ef287966fba2493ffb3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:20.385843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:20.424632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:20.443111 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982fc634537e4ef287966fba2493ffb3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:20.455920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982fc634537e4ef287966fba2493ffb3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:20.456614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982fc634537e4ef287966fba2493ffb3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:20.457212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982fc634537e4ef287966fba2493ffb3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:20.470781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982fc634537e4ef287966fba2493ffb3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:20.484192 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1242/2495] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:04:20 2022] DELETE /identity/v3/users/982fc634537e4ef287966fba2493ffb3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:20.489377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.490161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.490340 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.511620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:20.512879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:20.548067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:20.584076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1254/2496] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:04:20.594351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:20.595796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:20.654176 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489a5200-f14c-4152-bdcf-46c117c1b8b7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:20.657877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:20.658483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.658763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.659035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.668714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:20.670180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bded55-b770-4363-811d-d12c2e51c054 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:20.681991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1243/2497] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:20.965732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:20.967179 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:20.982958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b096-fe4a-4815-b22b-faa635663da0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:20.987685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:20.988268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:20.988587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:20.989530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] PATH_INFO: `/v3/projects/39a105e1d7c54c609b70064dcbfe04a4` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:20.993440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39a105e1d7c54c609b70064dcbfe04a4)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:20.995623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.045027 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39a105e1d7c54c609b70064dcbfe04a4', 'request_id': 'req-4380c692-237a-45b2-8f50-5b11a86a756a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:21.054293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39a105e1d7c54c609b70064dcbfe04a4', 'request_id': 'req-4380c692-237a-45b2-8f50-5b11a86a756a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:21.059852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39a105e1d7c54c609b70064dcbfe04a4', 'request_id': 'req-4380c692-237a-45b2-8f50-5b11a86a756a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:21.071517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] The token cache is being invalidate because project 39a105e1d7c54c609b70064dcbfe04a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:04:21.071836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:21.072509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-4380c692-237a-45b2-8f50-5b11a86a756a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:21.074131 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1255/2498] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:04:20 2022] DELETE /identity/v3/projects/39a105e1d7c54c609b70064dcbfe04a4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:21.079274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.079569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.079852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.099284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:21.100564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:21.162247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1244/2499] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:21.170831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.171123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.171404 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.191674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:21.192905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:21.201426 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1256/2500] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:04:21.205938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.215882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4af4ca-8520-4724-b525-ae5c7e498c3e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.220583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.221202 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.221489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.221769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.222476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:21.222780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.224423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.236988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1245/2501] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:04:21.242144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.251823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d5109-e1be-484b-b736-03f97c2ab06d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.257164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.257771 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.258054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.258329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.259005 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.260946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.298252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1257/2502] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:21.305023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.316985 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e9f9-7596-485d-8e71-b5298095ea20 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.360408 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:21.384043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.385098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.385579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.386082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.386777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.388766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.412426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1246/2503] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:21.417915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.428812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438dc5c-f413-4b95-a684-560b576af8c3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.434210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.434972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.435189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.435461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.437468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.444732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.456185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1258/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:21.460922 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.472566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1f0a6-edc5-47b3-8e7f-c045a8770c82 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.477695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.478305 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.478589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.478866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.479556 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.481191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffda76-afe4-485d-937e-029398bd7ed3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.491365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1247/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:21.495709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.506739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267e227-ae7f-4c74-a17a-834d38b8d0e9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.511711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.512410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.512727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.513007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] PATH_INFO: `/v3/projects/708774cd72554b18a7df3a26885df730/users/5c785f28306a43c48aac5c50aef389d3/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.529089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] RBAC: Authorizing `identity:create_grant(project_id=708774cd72554b18a7df3a26885df730, user_id=5c785f28306a43c48aac5c50aef389d3, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.531768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.545333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1259/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:04:21 2022] PUT /identity/v3/projects/708774cd72554b18a7df3a26885df730/users/5c785f28306a43c48aac5c50aef389d3/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:21.561163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.561449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.561728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.594661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] MFA Rules not processed for user `5c785f28306a43c48aac5c50aef389d3`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:21.595906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:21.667546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1248/2507] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:21.672064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.721619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:21.737223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466fa61a-fc26-431d-b660-05f51181750d admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.742471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.743113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.743397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.743668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.744381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.746343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.773217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1260/2508] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:04:21.777748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.788949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba70ac3-f6ef-4eb4-a125-fb379ec64eac None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.847880 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.848531 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.848817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.849097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.849786 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.852012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.873036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1249/2509] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:21 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:21.877060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.888235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991db72a-7a86-4f4e-b6a7-1f5ff486fda1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.892881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.893482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.893760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.894032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.894736 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.896323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.907191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1261/2510] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:21.912977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.923673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3aff6-46f0-4082-af3b-78021ac4b008 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:21.929328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:21.929991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:21.930279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:21.930561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] PATH_INFO: `/v3/projects/c5739ebba1e7402d8203a16daea82aed/users/7fea7ac62fd84a9191d5972141a06887/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:21.947641 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5739ebba1e7402d8203a16daea82aed, user_id=7fea7ac62fd84a9191d5972141a06887, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:21.950346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:21.962896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1250/2511] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:04:21 2022] PUT /identity/v3/projects/c5739ebba1e7402d8203a16daea82aed/users/7fea7ac62fd84a9191d5972141a06887/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:21.966848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:21.977201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3c34-fe02-42bd-a6ae-bf033c87da18 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.051319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.051939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.052223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.052545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.054108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.055712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c38ac1-4f12-4ae1-9359-921204caf214 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.066053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1262/2512] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:22.070664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.081263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81478466-64e4-42e8-a411-3e8df9306c98 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.086450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.087108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.087387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.087660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] PATH_INFO: `/v3/projects/c5739ebba1e7402d8203a16daea82aed/users/7fea7ac62fd84a9191d5972141a06887/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.089636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5739ebba1e7402d8203a16daea82aed, user_id=7fea7ac62fd84a9191d5972141a06887, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.092272 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.098579 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:22.104990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1251/2513] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:04:22 2022] PUT /identity/v3/projects/c5739ebba1e7402d8203a16daea82aed/users/7fea7ac62fd84a9191d5972141a06887/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:22.117368 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.117650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.117930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.145118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] MFA Rules not processed for user `7fea7ac62fd84a9191d5972141a06887`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:22.146327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:22.215112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1263/2514] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:04:22.227532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.229313 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:22.312163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d80e02-20f8-4165-bdf3-4b1b96b54c69 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.316502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.317275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.317837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.317837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.328353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.329791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.339118 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1252/2515] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:22.417041 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:22.426774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d00d9d-44b5-4fd7-af98-a5f7c2023c6a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:22.430448 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:22.431058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:22.431339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:22.431624 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:22.473353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:22.492064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:22.493581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbabef2-8a93-4413-920a-226b7154f82d service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:22.502537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1264/2516] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:32.335149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=35 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:32.498590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=9 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:33.511248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:33.574817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13559a3-8f7c-4e65-be97-e028fb371561 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:33.579382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:33.580017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:33.580301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:33.580608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] PATH_INFO: `/v3/users/5c785f28306a43c48aac5c50aef389d3` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:33.581751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c785f28306a43c48aac5c50aef389d3)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:33.584333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:33.631391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c785f28306a43c48aac5c50aef389d3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:33.639529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c785f28306a43c48aac5c50aef389d3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:33.640151 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c785f28306a43c48aac5c50aef389d3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:33.640788 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c785f28306a43c48aac5c50aef389d3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:33.654400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c785f28306a43c48aac5c50aef389d3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:33.667553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1253/2517] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:04:33 2022] DELETE /identity/v3/users/5c785f28306a43c48aac5c50aef389d3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:33.671985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:33.672529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:33.672636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:33.692201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:33.694565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:33.747828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:33.749290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:33.761612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1265/2518] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:33.771865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:33.774042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:33.825560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b379f0-06d3-4a2d-a92c-139dd23d616b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:33.829230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:33.829852 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:33.830136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:33.830407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:33.832168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:33.843942 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5125f7-c217-4aca-9d6b-532b581b1181 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:33.847542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:33.848152 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:33.848464 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:33.848739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:33.861289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:33.862715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:33.868653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:33.873350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1266/2519] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:04:33.913346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:33.915030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:33.927275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1254/2520] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:04:34.168211 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:34.170628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:34.186521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeca9dd-3e46-45a0-bcb2-57ca1733cb35 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:34.192746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:34.194346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.194609 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.194882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] PATH_INFO: `/v3/projects/708774cd72554b18a7df3a26885df730` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.196056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708774cd72554b18a7df3a26885df730)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:34.199331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:34.270850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708774cd72554b18a7df3a26885df730', 'request_id': 'req-be9c8fad-8de0-465a-bef5-7e509ec4daf1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.283061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708774cd72554b18a7df3a26885df730', 'request_id': 'req-be9c8fad-8de0-465a-bef5-7e509ec4daf1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.294147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708774cd72554b18a7df3a26885df730', 'request_id': 'req-be9c8fad-8de0-465a-bef5-7e509ec4daf1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.308894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] The token cache is being invalidate because project 708774cd72554b18a7df3a26885df730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:04:34.309185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.309893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.311558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1267/2521] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:04:34 2022] DELETE /identity/v3/projects/708774cd72554b18a7df3a26885df730 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:34.315539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:34.327147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d661078-e9d9-4648-ad42-cd3eafd603c5 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:34.399906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:34.400534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.400822 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.401101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] PATH_INFO: `/v3/users/7fea7ac62fd84a9191d5972141a06887` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.402350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fea7ac62fd84a9191d5972141a06887)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:34.404285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:34.451710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fea7ac62fd84a9191d5972141a06887'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.459499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fea7ac62fd84a9191d5972141a06887'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.460496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fea7ac62fd84a9191d5972141a06887'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.461184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fea7ac62fd84a9191d5972141a06887'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.473261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d7dfc193-f9a2-419d-925e-2956a031ea41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fea7ac62fd84a9191d5972141a06887'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.486393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1255/2522] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:04:34 2022] DELETE /identity/v3/users/7fea7ac62fd84a9191d5972141a06887 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:34.762220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:34.763818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:34.778351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c8fad-8de0-465a-bef5-7e509ec4daf1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:34.805900 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:34.841962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:34.842620 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.842903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.843190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] PATH_INFO: `/v3/projects/c5739ebba1e7402d8203a16daea82aed` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.844378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5739ebba1e7402d8203a16daea82aed)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:34.846392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:34.896218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5739ebba1e7402d8203a16daea82aed', 'request_id': 'req-f1046821-37f7-45a2-8741-c6671929c473'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.903564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5739ebba1e7402d8203a16daea82aed', 'request_id': 'req-f1046821-37f7-45a2-8741-c6671929c473'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.909064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5739ebba1e7402d8203a16daea82aed', 'request_id': 'req-f1046821-37f7-45a2-8741-c6671929c473'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.921150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] The token cache is being invalidate because project c5739ebba1e7402d8203a16daea82aed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:04:34.921456 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.922116 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f1046821-37f7-45a2-8741-c6671929c473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:04:34.923953 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1268/2523] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:04:34 2022] DELETE /identity/v3/projects/c5739ebba1e7402d8203a16daea82aed => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:04:34.931358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:34.931885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:34.931885 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:34.951702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:34.952916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:34.988661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:35.016576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1256/2524] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:35.024833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.025192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.025502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.045725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:35.046945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:35.055826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1269/2525] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:35.060273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.070503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b261ff1-e6f0-4cb4-8ce1-cac83f6b07f5 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.075177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.075792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.076077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.076376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.077091 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:04:35.077401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.079004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.091861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1257/2526] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:04:35.096332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.107087 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894d7533-b4a0-47b5-b5d5-ee7386d244cf None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.111889 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.112540 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.112821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.113096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.113749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.117024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.145567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1270/2527] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:04:35 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:35.150254 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.160573 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0272ccc-a4b0-4ef1-b2c7-9ef1a406b41b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.222223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.222768 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.223055 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.223333 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.224017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.226542 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.248568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1258/2528] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:04:35 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:04:35.253062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.264371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6267dc-58b1-498a-9c5c-9bdd31d69bf5 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.269582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.269708 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.269940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.270220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.271668 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.273917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.284618 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1271/2529] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:04:35.289377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.300630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6e08a7-1930-4bce-9e2c-879de5883998 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.305580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.306278 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.306627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.306969 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.307747 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.309444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baaac99-2841-4ead-b87b-05e569e66bf3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.330892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1259/2530] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:04:35.335805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.346220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070b757-6802-432f-b435-25b479a51b87 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.351564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.352225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.352550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.352813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] PATH_INFO: `/v3/projects/29a9462a18d24287b396b06231028d3a/users/62e8a7e7dba64398b0f6bdbc51c1e495/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.369474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a9462a18d24287b396b06231028d3a, user_id=62e8a7e7dba64398b0f6bdbc51c1e495, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.372192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.385600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1272/2531] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:04:35 2022] PUT /identity/v3/projects/29a9462a18d24287b396b06231028d3a/users/62e8a7e7dba64398b0f6bdbc51c1e495/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:04:35.390874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a2602-f64a-4ba7-989e-9596f081b3f9 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.391168 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a2602-f64a-4ba7-989e-9596f081b3f9 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.391449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a2602-f64a-4ba7-989e-9596f081b3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.410999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2b5a2602-f64a-4ba7-989e-9596f081b3f9 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:35.412896 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2b5a2602-f64a-4ba7-989e-9596f081b3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:35.471961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1260/2532] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:35.482345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:35.484068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:35.543599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932487b-aff8-4ce9-b03f-2c53684f4486 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:35.547594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:35.548223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:35.548533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:35.549068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:35.556644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:35.559235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:35.561943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:35.572970 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1273/2533] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:36.236606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb72486-5cee-4055-a603-197a721fd412 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:36.237070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb72486-5cee-4055-a603-197a721fd412 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:36.237214 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb72486-5cee-4055-a603-197a721fd412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:36.268048 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-bfb72486-5cee-4055-a603-197a721fd412 None None] MFA Rules not processed for user `62e8a7e7dba64398b0f6bdbc51c1e495`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:36.269668 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bfb72486-5cee-4055-a603-197a721fd412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:36.313102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:36.358422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1261/2534] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:36.371868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:36.373524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:36.448835 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8d0670-42f9-4934-b21a-afe5d4afc8d8 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:36.452355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:36.452971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:36.453248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:36.453516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:36.462582 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:36.464025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:36.472549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1274/2535] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:37.838105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221338-c68a-4815-bf4d-2e255bf9a0d4 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.838670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221338-c68a-4815-bf4d-2e255bf9a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.838670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60221338-c68a-4815-bf4d-2e255bf9a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.858842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-60221338-c68a-4815-bf4d-2e255bf9a0d4 None None] MFA Rules not processed for user `62e8a7e7dba64398b0f6bdbc51c1e495`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:37.860068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-60221338-c68a-4815-bf4d-2e255bf9a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:37.868037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1262/2536] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:37.904284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:37.905774 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:37.967722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b64190-1e08-451a-a7b3-bf2ebff186c2 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:37.971874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:37.972740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:37.973201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:37.973637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:37.976735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:38.051395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:38.053557 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631567e-2c61-4d66-81f3-f46154a70d8b service glance] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:38.064395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1275/2537] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:47.853124 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=14 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:48.059854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=82 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:04:55.249894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:55.251925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:55.252219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:55.302099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] MFA Rules not processed for user `62e8a7e7dba64398b0f6bdbc51c1e495`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:55.304913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:55.316166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1263/2538] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:55.770677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:55.771360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:55.771360 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:55.791982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] MFA Rules not processed for user `3f4c9c7f2ddd4403b13696794dfaab5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:04:55.793402 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:55.867909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1276/2539] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:04:58.001062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:58.012718 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a582ed-2510-4773-9954-47ca09ee6e94 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:58.017843 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:58.018470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:58.018758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:58.019036 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:58.021043 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:58.042410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:58.043879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:58.057005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1264/2540] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:04:58.230643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:04:58.232062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:58.245094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0848891-fd7a-46b5-86a9-d2aadfec3857 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:04:58.249567 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:04:58.250723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:04:58.251175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:04:58.251616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:04:58.255133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:04:58.267416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:04:58.270135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:04:58.292306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1277/2541] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:04:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:03.149323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.151400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:03.165482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbc09-9f2a-4a8e-a0b7-a3b6ea53bc2a service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.170242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.170925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.171212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.171491 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] PATH_INFO: `/v3/users/807855d862b146bdba009626f28d8d2f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.188670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=807855d862b146bdba009626f28d8d2f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.190704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.242182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807855d862b146bdba009626f28d8d2f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.252835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807855d862b146bdba009626f28d8d2f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.253659 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807855d862b146bdba009626f28d8d2f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.255074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807855d862b146bdba009626f28d8d2f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.271582 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-3a6a1c3e-d0b4-4b1f-8bf7-521f9aa6b7f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807855d862b146bdba009626f28d8d2f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.285548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1265/2542] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:03 2022] DELETE /identity/v3/users/807855d862b146bdba009626f28d8d2f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:03.471983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.485604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e61ab-9ddb-42b9-ab62-aad1067953dc service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.550478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.551138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.552072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.552366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] PATH_INFO: `/v3/projects/a5932111f3384861bfc778d561980e60` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.563471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5932111f3384861bfc778d561980e60)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.565528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.612639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5932111f3384861bfc778d561980e60', 'request_id': 'req-053b5a25-c4ec-474c-8cf3-569e3f9adc19'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.621406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5932111f3384861bfc778d561980e60', 'request_id': 'req-053b5a25-c4ec-474c-8cf3-569e3f9adc19'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.627210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5932111f3384861bfc778d561980e60', 'request_id': 'req-053b5a25-c4ec-474c-8cf3-569e3f9adc19'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.640191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] The token cache is being invalidate because project a5932111f3384861bfc778d561980e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:03.640514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.641207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053b5a25-c4ec-474c-8cf3-569e3f9adc19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:03.642752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1278/2543] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:03 2022] DELETE /identity/v3/projects/a5932111f3384861bfc778d561980e60 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:03.648052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.648394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.648961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.669860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:03.671094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:03.735178 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1266/2544] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:03.744077 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.744381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.744658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.765775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:03.767042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:03.776545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1279/2545] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:03.780739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.792449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7d62d4-6479-4678-8e9b-6d4111b1df6b None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.797315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.798044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.798394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.798999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.799874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:03.800380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.802823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.823698 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1267/2546] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:03.825886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.837329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b621ee3-1fc9-47f1-9bab-ea13530c1854 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.842434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.843060 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.843346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.843625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.844291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.846499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:03.851944 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:03.883723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1280/2547] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:03 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:03.887959 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:03.899974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5408f-e0c6-4339-9437-9ed6bfc09a93 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:03.943647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:03.976940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:03.977568 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:03.977848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:03.978122 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:03.978848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:03.980706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.005126 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1268/2548] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:03 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:04.009210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.021216 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf481f-97a1-48a5-a60a-57e1bd96c6b9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.027611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.028226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.028548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.028868 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.029560 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.031205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.042293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1281/2549] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:04.046416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.058270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbdebe-f5b5-4705-896b-bda4706b1a83 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.063067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.063681 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.063955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.064222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.064960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.066693 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6b4613-8e5d-4b4e-b37d-9107f365786c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.078892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1269/2550] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:04.082918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.094199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0a391-3b81-40da-8a57-d006f36eca5b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.099658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.100123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.100443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.100718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] PATH_INFO: `/v3/projects/ee13c73b881f45c98c37a16ff4a132ab/users/c2c286b2336146998742a3f2984e855c/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.119881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee13c73b881f45c98c37a16ff4a132ab, user_id=c2c286b2336146998742a3f2984e855c, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.122575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.138917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1282/2551] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:04 2022] PUT /identity/v3/projects/ee13c73b881f45c98c37a16ff4a132ab/users/c2c286b2336146998742a3f2984e855c/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:04.149794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.150364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.150364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.183605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] MFA Rules not processed for user `c2c286b2336146998742a3f2984e855c`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:04.185052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:04.279893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1270/2552] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:04.284155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.369817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f09ea-b9ec-4f41-a181-7cedb0c929b7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.376485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.377525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.377675 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.378002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.378628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.380849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.418995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1283/2553] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:04 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:04.420377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.432115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b890bb6-ab8b-452f-8232-4a20bca7af52 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.499541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.500170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.500503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.500779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.501493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.503422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.533986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1271/2554] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:04 2022] POST /identity/v3/users => generated 288 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:04.539147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.550164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72992e-9d30-4233-898a-990f7161e9fc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.556887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.557517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.557802 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.558100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.558801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.560452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.574143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1284/2555] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:04.579989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.592330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2240a-9cc9-42a9-9aac-d5a6af6044dd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.597320 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.598014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.598941 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.599348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] PATH_INFO: `/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e/users/541a2c584c6b458594d264a5003cdd0c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.617088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dcc5f42e8be4eebbe0e5963b3c1ed9e, user_id=541a2c584c6b458594d264a5003cdd0c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.620008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.634748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1272/2556] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:04 2022] PUT /identity/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e/users/541a2c584c6b458594d264a5003cdd0c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:04.638952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.652085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40447530-8c0f-42fe-a3af-30abd71216fc admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.729859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.730486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.730773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.731091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.731809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.733583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a793957-3e67-4bd3-85c3-bc382a1f2820 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.743998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1285/2557] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:04.748294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.753835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:04.761030 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676756e8-c897-47e2-861e-18facaf0a162 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.765763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.766427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.766732 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.767006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] PATH_INFO: `/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e/users/541a2c584c6b458594d264a5003cdd0c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.768998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dcc5f42e8be4eebbe0e5963b3c1ed9e, user_id=541a2c584c6b458594d264a5003cdd0c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.772722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:04.783527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1273/2558] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:04 2022] PUT /identity/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e/users/541a2c584c6b458594d264a5003cdd0c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:04.796240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.796564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.796846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.826562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] MFA Rules not processed for user `541a2c584c6b458594d264a5003cdd0c`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:04.827815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:04.836909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:04.902689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1286/2559] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:04.914450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:04.916512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:04.977171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cd30c-34f8-4bfd-9fd7-502db1304ba8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:04.980820 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:04.981486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:04.981722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:04.981995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:04.993954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:04.995409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:05.006285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1274/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:05.089886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:05.100590 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eae25e-1f0e-48a6-b249-e7189ae0d796 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:05.104759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:05.105461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:05.106556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:05.106911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:05.171858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:05.173327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:05.182899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1287/2561] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:05.753868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:05.764297 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a7a0-077a-4ade-b4f3-984e27be1363 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:05.768034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:05.768690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:05.768975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:05.769299 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:05.771997 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:05.873862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:05.876049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340340da-154d-469d-a185-ffc86307fe0a service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:05.907034 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1275/2562] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:15.178709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=53 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:15.902611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:16.787799 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:16.854939 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707dbf34-655a-4daf-ba0f-2396f6129c2b service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:16.859883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:16.860566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:16.860848 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:16.861123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] PATH_INFO: `/v3/users/c2c286b2336146998742a3f2984e855c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:16.862334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2c286b2336146998742a3f2984e855c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:16.864285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:16.930892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2c286b2336146998742a3f2984e855c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:16.931569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2c286b2336146998742a3f2984e855c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:16.946640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2c286b2336146998742a3f2984e855c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:16.956331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2c286b2336146998742a3f2984e855c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:16.957723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2c286b2336146998742a3f2984e855c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:16.985364 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1288/2563] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:16 2022] DELETE /identity/v3/users/c2c286b2336146998742a3f2984e855c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:16.987027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:16.987484 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:16.987617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:17.015191 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:17.016845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:17.101616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1276/2564] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:17.111335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:17.112786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:17.177422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca6ada-1a3d-4139-be69-d6f6e51334ff admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:17.181169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:17.181824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:17.182108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:17.182409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:17.192804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:17.194606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:17.205733 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1289/2565] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:17.504961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:17.506578 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:17.522433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c011a-3edf-4e9a-b0b1-936a4b35414e None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:17.529184 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:17.529897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:17.530265 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:17.530650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] PATH_INFO: `/v3/projects/ee13c73b881f45c98c37a16ff4a132ab` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:17.531900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee13c73b881f45c98c37a16ff4a132ab)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:17.533883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:17.714832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee13c73b881f45c98c37a16ff4a132ab', 'request_id': 'req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.723720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee13c73b881f45c98c37a16ff4a132ab', 'request_id': 'req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.738737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee13c73b881f45c98c37a16ff4a132ab', 'request_id': 'req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.755848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] The token cache is being invalidate because project ee13c73b881f45c98c37a16ff4a132ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:17.756142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.756861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.758657 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1277/2566] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:17 2022] DELETE /identity/v3/projects/ee13c73b881f45c98c37a16ff4a132ab => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:17.766476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:17.777814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8626189-811c-40a7-a7b2-b9232dc09459 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:17.815339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:17.874964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:17.876213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:17.876213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:17.876213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] PATH_INFO: `/v3/users/541a2c584c6b458594d264a5003cdd0c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:17.877511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=541a2c584c6b458594d264a5003cdd0c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:17.880096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:17.941121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541a2c584c6b458594d264a5003cdd0c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.943003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541a2c584c6b458594d264a5003cdd0c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.953071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541a2c584c6b458594d264a5003cdd0c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.962304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541a2c584c6b458594d264a5003cdd0c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.963078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541a2c584c6b458594d264a5003cdd0c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:17.981020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1290/2567] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:17 2022] DELETE /identity/v3/users/541a2c584c6b458594d264a5003cdd0c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:18.219769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.221833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:18.299220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.299566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b09eb63-7ce4-4c3b-9cfa-ae4fac33de85 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.300928 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:18.350793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:18.365195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.365971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.367402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.367682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.369580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:18.398429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee1e4b-577c-4607-8d86-f5fd70b4b5f1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.403724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.404580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.404921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.405277 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] PATH_INFO: `/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.406509 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dcc5f42e8be4eebbe0e5963b3c1ed9e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.408591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.458909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.460921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbddcc4-f6df-4baa-9e4b-c2bb466adc2a service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.467339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:18.471157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1278/2568] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:18.472301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dcc5f42e8be4eebbe0e5963b3c1ed9e', 'request_id': 'req-98b0487c-a227-4873-acc9-5bab610e89c1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:18.482094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dcc5f42e8be4eebbe0e5963b3c1ed9e', 'request_id': 'req-98b0487c-a227-4873-acc9-5bab610e89c1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:18.490403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dcc5f42e8be4eebbe0e5963b3c1ed9e', 'request_id': 'req-98b0487c-a227-4873-acc9-5bab610e89c1'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:18.505670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] The token cache is being invalidate because project 0dcc5f42e8be4eebbe0e5963b3c1ed9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:18.506021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:18.506851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98b0487c-a227-4873-acc9-5bab610e89c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:18.508871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1291/2569] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:18 2022] DELETE /identity/v3/projects/0dcc5f42e8be4eebbe0e5963b3c1ed9e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:18.514077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.514622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.515067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.542077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:18.543421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:18.613950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1279/2570] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:18.623325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.623591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.623937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.646553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:18.647804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:18.670383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1292/2571] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:18.675152 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.687898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf520f3-71c4-439c-80df-5bec58d58b1c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.695379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.696018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.696873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.696961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.697767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:18.698115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.700530 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.717507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1280/2572] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:18.723700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.748117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22991d-d1ff-44d5-bd18-1eaa0f10cd1d None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.752820 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.753442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.753727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.754815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.755558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.757599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.797375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1293/2573] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:18 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:18.802015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.817211 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd13f4-c110-49fd-8963-080764077c2e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.914741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.915638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.915995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.916361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.917114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.920181 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.956526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1281/2574] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:18 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:18.957823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:18.969769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb52f713-afd0-45c2-aaa5-0f7d19c0a896 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:18.975361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:18.975986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:18.976268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:18.976616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:18.977451 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:18.979018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:18.994213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1294/2575] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:18.995744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.007607 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da61a7-b400-42f3-8375-bdaf61fd30cc admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.012735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.013432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.013784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.014129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.014893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.016595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044059b0-7dae-40ff-a211-33247ea45564 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.018139 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:19.032410 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1282/2576] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:19.034080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.047575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b9afd-8900-432a-9e36-634fbdb9f2e6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.054053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.054749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.055016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.055297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] PATH_INFO: `/v3/projects/12af94af1dd34026ada996f394f15efd/users/1a5ba3081b2048b4b703e7016786a517/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.071121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12af94af1dd34026ada996f394f15efd, user_id=1a5ba3081b2048b4b703e7016786a517, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.073829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.090759 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1295/2577] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:19 2022] PUT /identity/v3/projects/12af94af1dd34026ada996f394f15efd/users/1a5ba3081b2048b4b703e7016786a517/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:19.102017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.102310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.102644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.139762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] MFA Rules not processed for user `1a5ba3081b2048b4b703e7016786a517`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:19.141188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:19.216713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1283/2578] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:19.221089 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.313497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb577943-ebdb-43bb-8d49-f8079558d5d2 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.319482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.320104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.320419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.320692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.321348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.323242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.353138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1296/2579] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:19 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:19.357993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.369346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf9cee-550e-40ce-9af6-ad27e48f08aa None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.431858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.432448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.432735 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.433142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.433963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.436206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.457933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1284/2580] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:19 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:19.461974 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.472046 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25c1cf3-76bc-43b9-afa4-7317417cfc93 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.477419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.478104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.478388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.478662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.479447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.481473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.493559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1297/2581] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:19.497712 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.508436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a91370-ae79-4ec8-8db3-845b1c94d99c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.520210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.523785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.524112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.524397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] PATH_INFO: `/v3/projects/a52493491a9f494e9eaa6a5cd327292c/users/3bd170057fde45b1931a5d4f8df4aa24/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.545212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52493491a9f494e9eaa6a5cd327292c, user_id=3bd170057fde45b1931a5d4f8df4aa24, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.547872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.565871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1285/2582] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:19 2022] PUT /identity/v3/projects/a52493491a9f494e9eaa6a5cd327292c/users/3bd170057fde45b1931a5d4f8df4aa24/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:19.567677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.579521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d898f5ad-22a4-41ec-a26b-38b29d275a39 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.638519 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:19.658949 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.659584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.659866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.660208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.661493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.663253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8843995-e630-4d4d-830a-a729caab5215 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.676387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1298/2583] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:19.679296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.690493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f0ecd-0d00-43a7-8d8f-b244001d6bfd admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.695697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.696399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.696686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.696962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] PATH_INFO: `/v3/projects/a52493491a9f494e9eaa6a5cd327292c/users/3bd170057fde45b1931a5d4f8df4aa24/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.699066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a52493491a9f494e9eaa6a5cd327292c, user_id=3bd170057fde45b1931a5d4f8df4aa24, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.701860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.721948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1286/2584] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:19 2022] PUT /identity/v3/projects/a52493491a9f494e9eaa6a5cd327292c/users/3bd170057fde45b1931a5d4f8df4aa24/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:19.734090 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.734384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.734666 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.765445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] MFA Rules not processed for user `3bd170057fde45b1931a5d4f8df4aa24`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:19.766811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:19.838465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1299/2585] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:19.849567 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:19.851352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:19.896102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:19.912897 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfecdb4-43ff-49b8-8ecd-17f0c5f95779 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:19.916628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:19.917251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:19.917553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:19.917833 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:19.979600 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:19.980993 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:19.990508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1287/2586] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:21.859051 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:21.860610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:21.932723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463783-6eff-4ead-a90e-92f78e93a4c5 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:21.937566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:21.938291 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:21.938569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:21.938841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] PATH_INFO: `/v3/users/a1930e444b5b47ea8e606c7ee4da600f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:21.940130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1930e444b5b47ea8e606c7ee4da600f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:21.942909 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:21.998233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1930e444b5b47ea8e606c7ee4da600f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:21.999199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1930e444b5b47ea8e606c7ee4da600f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.007382 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1930e444b5b47ea8e606c7ee4da600f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.014904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1930e444b5b47ea8e606c7ee4da600f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.015601 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8ce114e6-5ad7-475e-b1fe-8698ae0a744d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1930e444b5b47ea8e606c7ee4da600f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.017647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:22.037039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1300/2587] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:21 2022] DELETE /identity/v3/users/a1930e444b5b47ea8e606c7ee4da600f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:22.218081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.230096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d636a-01ea-4ce1-be75-111a263cf3e0 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.298045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.298045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.298045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.298045 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] PATH_INFO: `/v3/projects/c791c03c7d7b40fcb2e0ffad05d78172` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.298807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c791c03c7d7b40fcb2e0ffad05d78172)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.298807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.349444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c791c03c7d7b40fcb2e0ffad05d78172', 'request_id': 'req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.359209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c791c03c7d7b40fcb2e0ffad05d78172', 'request_id': 'req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.368133 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c791c03c7d7b40fcb2e0ffad05d78172', 'request_id': 'req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.381206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] The token cache is being invalidate because project c791c03c7d7b40fcb2e0ffad05d78172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:22.381500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.382167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-c4be7f41-bbd4-4b7f-8cb6-e83a4d00e4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:22.383678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1288/2588] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:22 2022] DELETE /identity/v3/projects/c791c03c7d7b40fcb2e0ffad05d78172 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:22.388824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.389053 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.389331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.411998 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:22.413257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:22.477289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1301/2589] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:22.484832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.485113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.485383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.507108 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:22.508308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:22.516780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1289/2590] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:22.521662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.532846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9bad1-6f7a-4976-91f9-98b19dfc06ea None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.539529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.540055 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.540520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.540798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.541511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:22.541829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.543433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.556896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1302/2591] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:22.561447 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.570846 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae29c6b-9d0c-4906-a300-51d62266b72d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.575574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.576182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.576503 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.576765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.577520 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.579591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.596362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:22.619075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1290/2592] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:22 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:22.620249 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.631750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67349c27-8059-49f4-b2de-4f31036dc5cd admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.690743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.692210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.692720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.693023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.693716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.695635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.718367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1303/2593] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:22 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:22.722586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.733501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e74fab-e30e-48c5-a372-6963f90fcadb admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.738639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.739606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.739900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.740172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.740903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.742500 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.752918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1291/2594] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:22.758471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.768706 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39ca06-9320-4302-abb7-ab275079166b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.773506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.774113 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.774398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.774678 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.775367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.777016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e04cc6-107c-40ed-a5de-606326cc9402 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.787394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1304/2595] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:22.791541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.802499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400330f3-0079-4603-a4ca-afe0757caa28 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:22.807234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:22.807953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.808273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.808612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] PATH_INFO: `/v3/projects/b3d3cbb3c86a4227a8fdd5c2bf117c9e/users/c14e8c3e9e4944fbb37a4a633ce62734/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.823042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d3cbb3c86a4227a8fdd5c2bf117c9e, user_id=c14e8c3e9e4944fbb37a4a633ce62734, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:22.825717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:22.838084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1292/2596] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:22 2022] PUT /identity/v3/projects/b3d3cbb3c86a4227a8fdd5c2bf117c9e/users/c14e8c3e9e4944fbb37a4a633ce62734/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:22.849972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e031ef-86e3-431c-9c06-1f20b374401a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:22.850259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e031ef-86e3-431c-9c06-1f20b374401a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:22.850534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e031ef-86e3-431c-9c06-1f20b374401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:22.879419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c4e031ef-86e3-431c-9c06-1f20b374401a None None] MFA Rules not processed for user `c14e8c3e9e4944fbb37a4a633ce62734`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:22.880672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c4e031ef-86e3-431c-9c06-1f20b374401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:22.919112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:22.948115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1305/2597] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:22.959355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:22.961062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:23.034809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823c01-f0e7-4505-bbba-834ce5d1a34c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:23.041040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:23.041669 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:23.041952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:23.042226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:23.051875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:23.053474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:23.062770 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1293/2598] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:24.655391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:24.656284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:24.656596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:24.713288 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:24.714597 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:24.795019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1306/2599] 10.222.0.52 () {58 vars in 1009 bytes} [Thu Aug 4 21:05:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:24.849658 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:24.851790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:24.943889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6013-c459-40b7-8245-8341a08faf1d service nova] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:24.949240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:24.949858 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:24.950200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:24.950446 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:24.959466 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:24.961079 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:24.969102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1294/2600] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:05:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:05:30.056380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:30.058547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:30.132652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dd2807-8dc8-451f-b3cb-0c7762edaa03 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:30.136355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:30.137000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:30.137474 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:30.137688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:30.139498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:30.220636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:30.222120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1286a56-b933-4fc9-952c-e2d96e17ad8d service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:30.230072 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:30.235822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1307/2601] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:34.965860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:INSERT=2,keystone:SELECT=92 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:41.119984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:41.197875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848da0cb-beb5-4075-8a3c-88d7985ce0d1 service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:41.204661 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:41.205315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:41.205596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:41.205868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] PATH_INFO: `/v3/users/c14e8c3e9e4944fbb37a4a633ce62734` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:41.207691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14e8c3e9e4944fbb37a4a633ce62734)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:41.209807 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:41.284379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14e8c3e9e4944fbb37a4a633ce62734'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:41.295382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14e8c3e9e4944fbb37a4a633ce62734'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:41.296075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14e8c3e9e4944fbb37a4a633ce62734'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:41.297245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14e8c3e9e4944fbb37a4a633ce62734'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:41.333311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14e8c3e9e4944fbb37a4a633ce62734'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:41.350454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1295/2602] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:41 2022] DELETE /identity/v3/users/c14e8c3e9e4944fbb37a4a633ce62734 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:41.356787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:41.356787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:41.356787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:41.379386 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:41.381069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:41.453559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1308/2603] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:41.465162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:41.466714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:41.538359 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71202694-a32a-4fa3-a208-a5cfa47ca743 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:41.544791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:41.545428 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:41.545714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:41.546114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:41.577574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:41.580471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb33e22-befc-4239-90b2-f95107ebe476 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:41.595982 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1296/2604] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:41.917361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:41.919004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:41.940581 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2ce76-2713-4c64-97f9-d6e6aa9dc03c None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:41.947014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:41.948361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:41.949009 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:41.949593 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] PATH_INFO: `/v3/projects/b3d3cbb3c86a4227a8fdd5c2bf117c9e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:41.952878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d3cbb3c86a4227a8fdd5c2bf117c9e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:41.957809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.019391 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d3cbb3c86a4227a8fdd5c2bf117c9e', 'request_id': 'req-3e8969c3-5d83-4660-ae7d-d84820661c1c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:42.029323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d3cbb3c86a4227a8fdd5c2bf117c9e', 'request_id': 'req-3e8969c3-5d83-4660-ae7d-d84820661c1c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:42.036292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d3cbb3c86a4227a8fdd5c2bf117c9e', 'request_id': 'req-3e8969c3-5d83-4660-ae7d-d84820661c1c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:42.048660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] The token cache is being invalidate because project b3d3cbb3c86a4227a8fdd5c2bf117c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:42.049097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:42.049818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-3e8969c3-5d83-4660-ae7d-d84820661c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:42.051954 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1309/2605] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:41 2022] DELETE /identity/v3/projects/b3d3cbb3c86a4227a8fdd5c2bf117c9e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:42.056912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.057498 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.058072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.079586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:42.080893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:42.100912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:42.148532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1297/2606] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:42.155858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.156156 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.156475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.177506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:42.178725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:42.187294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1310/2607] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:42.195056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.205831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8f8df-8e52-41a8-a5cb-616053267506 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.214113 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.214763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.215047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.215328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.216038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:42.216354 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.217991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.232977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1298/2608] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:42.238876 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.249577 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87243d-03b5-4687-9e82-8f8203159cde None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.254407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.255085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.255367 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.255641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.256299 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.258345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.289803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1311/2609] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:42 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:42.293884 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.305360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6cf94-3a66-4155-969f-3dd7dfb73648 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.366666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.367285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.367571 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.367851 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.368536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.370453 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.390436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1299/2610] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:42 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:42.395025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.405431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953b59b-2b16-4daa-880a-e7cf82029e8c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.410529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.411326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.411605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.411873 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.412619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.414207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.424436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1312/2611] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:42.429186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.438957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7bb5da-1e83-4aa2-9c70-5cb2e1b2c599 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.444026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.444664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.444942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.445215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.445921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.447501 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ba8c3-108c-4c4f-8b2f-ae5a4b11d447 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.458263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1300/2612] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:42.462587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.473411 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3e655-e74d-44b6-a3a8-176de2d68bde admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.478172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.478823 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.479104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.479393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] PATH_INFO: `/v3/projects/c9888296c5e440c8b37cee6b2f596c9c/users/ae5fb2f35d7d41a3b7decd94e02cb40f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.494696 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9888296c5e440c8b37cee6b2f596c9c, user_id=ae5fb2f35d7d41a3b7decd94e02cb40f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.497403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.510770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1313/2613] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:42 2022] PUT /identity/v3/projects/c9888296c5e440c8b37cee6b2f596c9c/users/ae5fb2f35d7d41a3b7decd94e02cb40f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:42.515918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da767a-9022-48ce-9694-5ee0de3b4b33 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.516284 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da767a-9022-48ce-9694-5ee0de3b4b33 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.516614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da767a-9022-48ce-9694-5ee0de3b4b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.536809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-99da767a-9022-48ce-9694-5ee0de3b4b33 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:42.538016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-99da767a-9022-48ce-9694-5ee0de3b4b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:42.601026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1301/2614] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:42.608422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:42.610221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:42.632320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:42.669085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa8f5-5241-412e-9b78-9d679620b7ef admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:42.672679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:42.673538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:42.673817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:42.674086 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:42.682899 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:42.684427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:42.693982 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1314/2615] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:43.321150 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.321644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.321782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.353222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] MFA Rules not processed for user `ae5fb2f35d7d41a3b7decd94e02cb40f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:43.354552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:43.373489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:43.424774 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1302/2616] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:43.429270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.439811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a471077-e260-4906-9f0a-0ad95c77c0a5 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.444504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.445274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.445805 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.446161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.447056 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.449059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.478172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1315/2617] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:43 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:43.482200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.495899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800befe7-7b69-4660-aefd-c93698b29e5c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.557809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.558425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.558706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.558981 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.559646 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.561605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.583323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1303/2618] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:43 2022] POST /identity/v3/users => generated 308 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:43.587371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.598311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4540c-133c-4c65-8278-5e23b825dfc4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.602995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.603604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.603885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.604160 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.604886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.606473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.617300 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1316/2619] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:43.621618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.633382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b16e3-5b43-4625-b006-d8a00cd1c337 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.638381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.638834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.639119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.639397 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] PATH_INFO: `/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a/users/42306f24e8c94c7ea022d6cb5130a58c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.658666 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed60dcbee1a4b5191feb6548f7c3c7a, user_id=42306f24e8c94c7ea022d6cb5130a58c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.662587 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.675416 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1304/2620] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:43 2022] PUT /identity/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a/users/42306f24e8c94c7ea022d6cb5130a58c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:43.679971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.689915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e87b489-5783-4759-bb48-a84811d61c09 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.745353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.745963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.746375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.746650 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.747326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.748938 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231021-3ef5-4e0d-91a2-7f11459e1096 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.759105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1317/2621] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:43.764103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:43.775144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de914be8-8aa7-4ce6-9785-4f2130fd7655 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:43.780078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:43.780779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:43.781062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:43.781351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] PATH_INFO: `/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a/users/42306f24e8c94c7ea022d6cb5130a58c/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:43.783371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed60dcbee1a4b5191feb6548f7c3c7a, user_id=42306f24e8c94c7ea022d6cb5130a58c, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:43.786350 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:43.796717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1305/2622] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:43 2022] PUT /identity/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a/users/42306f24e8c94c7ea022d6cb5130a58c/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:44.440115 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:44.440651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:44.440769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:44.473361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] MFA Rules not processed for user `42306f24e8c94c7ea022d6cb5130a58c`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:44.474712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:44.549358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1318/2623] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:45.611154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:45.675330 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ffba7-123a-4359-9b34-8b1e1a6a0e16 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:45.680115 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:45.680798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:45.681077 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:45.681347 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] PATH_INFO: `/v3/users/ae5fb2f35d7d41a3b7decd94e02cb40f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:45.682716 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae5fb2f35d7d41a3b7decd94e02cb40f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:45.684622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:45.697084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:45.731723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae5fb2f35d7d41a3b7decd94e02cb40f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:45.738973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae5fb2f35d7d41a3b7decd94e02cb40f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:45.739742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae5fb2f35d7d41a3b7decd94e02cb40f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:45.740547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae5fb2f35d7d41a3b7decd94e02cb40f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:45.752682 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae5fb2f35d7d41a3b7decd94e02cb40f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:45.765162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1306/2624] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:45 2022] DELETE /identity/v3/users/ae5fb2f35d7d41a3b7decd94e02cb40f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:45.943898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:45.956494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5096519-db39-4662-b3bf-03f2a1fbc619 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:45.965735 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:46.018547 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.019956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.020313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.020627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] PATH_INFO: `/v3/projects/c9888296c5e440c8b37cee6b2f596c9c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.021791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9888296c5e440c8b37cee6b2f596c9c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.023784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.071037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9888296c5e440c8b37cee6b2f596c9c', 'request_id': 'req-f5bd3e6d-3330-4117-b99f-94e563f1089a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.078344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9888296c5e440c8b37cee6b2f596c9c', 'request_id': 'req-f5bd3e6d-3330-4117-b99f-94e563f1089a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.086481 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9888296c5e440c8b37cee6b2f596c9c', 'request_id': 'req-f5bd3e6d-3330-4117-b99f-94e563f1089a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.100849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] The token cache is being invalidate because project c9888296c5e440c8b37cee6b2f596c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:46.101260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.102241 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.104684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1319/2625] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:45 2022] DELETE /identity/v3/projects/c9888296c5e440c8b37cee6b2f596c9c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:46.110053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.111584 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:46.125095 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03816a9-6ee3-40ee-9e5a-43bfaa1564d8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.181860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.182505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.182794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.183072 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] PATH_INFO: `/v3/users/42306f24e8c94c7ea022d6cb5130a58c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.184247 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42306f24e8c94c7ea022d6cb5130a58c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.186216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.232185 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42306f24e8c94c7ea022d6cb5130a58c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.239481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42306f24e8c94c7ea022d6cb5130a58c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.240172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42306f24e8c94c7ea022d6cb5130a58c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.240860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42306f24e8c94c7ea022d6cb5130a58c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.252872 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-d1f33fa9-7caa-4c0b-931c-41caf5f2f0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42306f24e8c94c7ea022d6cb5130a58c'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.265782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1307/2626] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:46 2022] DELETE /identity/v3/users/42306f24e8c94c7ea022d6cb5130a58c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:46.447069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.448621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:46.468522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd3e6d-3330-4117-b99f-94e563f1089a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.524852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.525545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.525869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.526165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] PATH_INFO: `/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.527850 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed60dcbee1a4b5191feb6548f7c3c7a)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.529562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.574842 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:46.578860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed60dcbee1a4b5191feb6548f7c3c7a', 'request_id': 'req-82450eaa-0dbb-47e9-bdb1-9216266159f2'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.586853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed60dcbee1a4b5191feb6548f7c3c7a', 'request_id': 'req-82450eaa-0dbb-47e9-bdb1-9216266159f2'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.592681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed60dcbee1a4b5191feb6548f7c3c7a', 'request_id': 'req-82450eaa-0dbb-47e9-bdb1-9216266159f2'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.603495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] The token cache is being invalidate because project 6ed60dcbee1a4b5191feb6548f7c3c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:46.603856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.604520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-82450eaa-0dbb-47e9-bdb1-9216266159f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:46.606015 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1320/2627] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:46 2022] DELETE /identity/v3/projects/6ed60dcbee1a4b5191feb6548f7c3c7a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:46.610919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.611217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.611794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.632099 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:46.633338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:46.637918 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:46.697109 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1308/2628] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:46.704858 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.705148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.705428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.727501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:46.728745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:46.738173 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1321/2629] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:46.741649 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.751512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b526c4-a7c1-4afa-bebb-c64f6b25914d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.757057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.757676 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.757992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.758271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.758960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:46.759270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.760900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.774368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1309/2630] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:46.778841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.789003 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae25e21-c715-4ffc-b58b-103f289e0b4e None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.794166 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.794794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.795075 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.795345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.796008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.797937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.832874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1322/2631] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:46 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:46.834548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.845276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d64a2b-9fce-452a-b49e-83d3a43b643c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.910434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.911283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.911554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.911834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.912536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.914477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.935835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1310/2632] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:46 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:46.939948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.950983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da2e47-d4e3-44fc-b977-e1c27a9da4d1 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.955641 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.956225 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.956531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.956801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.957679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.959261 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:46.969325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1323/2633] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:46.973989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:46.983933 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3323b82-01de-4b5c-b2a1-5012a5e8210a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:46.989438 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:46.990074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:46.990384 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:46.990660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:46.991372 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:46.993013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a8ef6-589c-42fa-92b4-2266404604c5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.003782 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1311/2634] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:47.007894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.018253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e9307-35b4-4110-aac8-a5285aecebf3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.023418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.024078 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.024476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.024662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] PATH_INFO: `/v3/projects/3270f57ff2354531bfb34d54bafc1271/users/5b5ea8dc6b05425992c915169f2a3295/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.040725 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=3270f57ff2354531bfb34d54bafc1271, user_id=5b5ea8dc6b05425992c915169f2a3295, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.043297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.057365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1324/2635] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:47 2022] PUT /identity/v3/projects/3270f57ff2354531bfb34d54bafc1271/users/5b5ea8dc6b05425992c915169f2a3295/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:47.067726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f00e6-1160-4372-bcf6-02fc4ec991a6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.068014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f00e6-1160-4372-bcf6-02fc4ec991a6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.068286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811f00e6-1160-4372-bcf6-02fc4ec991a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.096130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-811f00e6-1160-4372-bcf6-02fc4ec991a6 None None] MFA Rules not processed for user `5b5ea8dc6b05425992c915169f2a3295`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:47.098046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-811f00e6-1160-4372-bcf6-02fc4ec991a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:47.165105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1312/2636] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:47.169617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.238801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8eaae-3bed-4850-968c-567ab75e7ade admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.243672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.244459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.244744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.245035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.245699 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.246066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.246360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.246645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.247578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.255986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:47.270200 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] MFA Rules not processed for user `1fb00a6bca4f48468574ff17083e1fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:47.271392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:47.276783 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1325/2637] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:47 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:47.281004 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.330859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1313/2638] 10.222.0.52 () {58 vars in 967 bytes} [Thu Aug 4 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:47.336165 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:47.344470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.348367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:47.348968 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45b29f-5182-400c-9383-7596b4c378e3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.355045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.355757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.356108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.356482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.357149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.359158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.382311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1326/2639] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:47 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:47.385860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.395495 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042735b-aa19-48db-8f4c-f8ef5ef8ee3c admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.400512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.401161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.401467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.401767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.402455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.404054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.415028 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1327/2640] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:47.419233 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.420207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24adb9-3bab-4c2f-89c0-329cd8c65df5 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.429676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e1b49-ba85-4521-914d-3fb38d39aecb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.434307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.434978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.435264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.435541 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] PATH_INFO: `/v3/projects/0b662a9a55d14dbe93a13e982e157186/users/2de30811007046d68634894a7c2723c5/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.450870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b662a9a55d14dbe93a13e982e157186, user_id=2de30811007046d68634894a7c2723c5, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.453647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.466956 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1328/2641] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:47 2022] PUT /identity/v3/projects/0b662a9a55d14dbe93a13e982e157186/users/2de30811007046d68634894a7c2723c5/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:47.470910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.482154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '352be9523cf1454a86c16640a87ece9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.482778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.483061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.483336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.541958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=100 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:47.546097 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2ec79-a2d7-41f5-9c29-b2f375ed3cf8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.546474 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.548610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.551538 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.552306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.552709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.553021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.553827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.555631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7c31c8-2744-41fd-bc8f-04910bca33c5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.558106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1314/2642] 10.222.0.52 () {60 vars in 1384 bytes} [Thu Aug 4 21:05:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 21:05:47.569598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1329/2643] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:47.570756 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.582877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7709c18d-7b41-4546-8e63-f0ee86f4e3ce service swift] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.588606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.589652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.590021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.590362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] PATH_INFO: `/v3/projects/0b662a9a55d14dbe93a13e982e157186/users/2de30811007046d68634894a7c2723c5/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.592787 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b662a9a55d14dbe93a13e982e157186, user_id=2de30811007046d68634894a7c2723c5, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.595510 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.608566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1315/2644] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:47 2022] PUT /identity/v3/projects/0b662a9a55d14dbe93a13e982e157186/users/2de30811007046d68634894a7c2723c5/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:47.621217 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.622032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.622403 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.651787 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] MFA Rules not processed for user `2de30811007046d68634894a7c2723c5`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:47.653153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:47.727559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1330/2645] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:47 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:47.740173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:47.742076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:47.813898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c531e3-ca71-4b93-a4c2-dc46f384b653 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:47.818955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:47.819580 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:47.819861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:47.820203 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:47.832368 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:47.833962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:47.844444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1316/2646] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:47 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:48.201417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:48.220138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2b6c-e02f-445e-b617-8b8bb3a0d709 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:48.225867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de30811007046d68634894a7c2723c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b662a9a55d14dbe93a13e982e157186', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:48.226511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.226813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.227088 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.227781 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:48.228084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:48.229697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:48.241908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1331/2647] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:48.246189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:48.256761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b123125-c747-4eed-b61f-e3011784028c service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:48.262273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de30811007046d68634894a7c2723c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b662a9a55d14dbe93a13e982e157186', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:48.262961 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.263252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.263533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.264206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:48.266342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:48.296051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1317/2648] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:48 2022] POST /identity/v3/projects => generated 352 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:48.300514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:48.310816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c9018-ed45-45a6-81b6-d8402f96557d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:48.323119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:48.370824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de30811007046d68634894a7c2723c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b662a9a55d14dbe93a13e982e157186', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:48.371366 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.371652 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.371929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.372677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:48.372988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:48.374571 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:48.387229 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1332/2649] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:48.424171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:48.434614 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f418051-f9e8-4b16-ac1b-e58d7805f19d tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:48.439251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2de30811007046d68634894a7c2723c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b662a9a55d14dbe93a13e982e157186', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:48.439879 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.440166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.440488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] PATH_INFO: `/v3/projects/89a2d7ac31bc44cfb6350045ef6644e6` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.442188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorizing `identity:delete_project(project_id=89a2d7ac31bc44cfb6350045ef6644e6)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:48.444094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:48.489861 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a2d7ac31bc44cfb6350045ef6644e6', 'request_id': 'req-6009a808-f456-47cc-bacc-1253e816299f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.496980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a2d7ac31bc44cfb6350045ef6644e6', 'request_id': 'req-6009a808-f456-47cc-bacc-1253e816299f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.502748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a2d7ac31bc44cfb6350045ef6644e6', 'request_id': 'req-6009a808-f456-47cc-bacc-1253e816299f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.521338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] The token cache is being invalidate because project 89a2d7ac31bc44cfb6350045ef6644e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:48.521634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.522605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6009a808-f456-47cc-bacc-1253e816299f tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.524164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1318/2650] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:48 2022] DELETE /identity/v3/projects/89a2d7ac31bc44cfb6350045ef6644e6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:48.717280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:48.718965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:48.782500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474a381a-8a58-4973-822c-a61ba00e30b3 tempest-VolumeQuotaClassesTest-296614715 tempest-VolumeQuotaClassesTest-296614715-project-admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:48.839326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:48.839977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.840263 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.840661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] PATH_INFO: `/v3/users/5b5ea8dc6b05425992c915169f2a3295` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.841765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5ea8dc6b05425992c915169f2a3295)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:48.843689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:48.889525 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5ea8dc6b05425992c915169f2a3295'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.890164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5ea8dc6b05425992c915169f2a3295'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.898191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5ea8dc6b05425992c915169f2a3295'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.903305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:48.907506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5ea8dc6b05425992c915169f2a3295'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.908289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5ea8dc6b05425992c915169f2a3295'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:48.927307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1333/2651] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:48 2022] DELETE /identity/v3/users/5b5ea8dc6b05425992c915169f2a3295 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:48.931791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:48.932257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:48.932407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:48.953402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:48.954599 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:48.996800 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:49.017729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1319/2652] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:49.028685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:49.030147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:49.092247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99ad7f-e248-4633-8981-7b9aa74025be admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:49.097575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:49.097575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:49.097575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:49.097941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:49.107645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:49.109100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:49.120773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1334/2653] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:49.420729 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:49.422315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:49.436371 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5effe2a-19e7-47c6-aa3f-05f8245f1425 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:49.442910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:49.443875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:49.444255 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:49.444609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] PATH_INFO: `/v3/projects/3270f57ff2354531bfb34d54bafc1271` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:49.446063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3270f57ff2354531bfb34d54bafc1271)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:49.448452 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:49.501592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3270f57ff2354531bfb34d54bafc1271', 'request_id': 'req-be468b79-311b-4a5b-8c22-45fa0f2a571e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.510166 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3270f57ff2354531bfb34d54bafc1271', 'request_id': 'req-be468b79-311b-4a5b-8c22-45fa0f2a571e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.516289 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3270f57ff2354531bfb34d54bafc1271', 'request_id': 'req-be468b79-311b-4a5b-8c22-45fa0f2a571e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.530518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] The token cache is being invalidate because project 3270f57ff2354531bfb34d54bafc1271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:49.530948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.531667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.537487 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1320/2654] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:49 2022] DELETE /identity/v3/projects/3270f57ff2354531bfb34d54bafc1271 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:49.538681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:49.550399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac3b7-b417-4155-a644-bc6f556be24b service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:49.568842 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:49.575322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:49.625425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:49.626069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:49.626346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:49.626628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] PATH_INFO: `/v3/users/2de30811007046d68634894a7c2723c5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:49.627770 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de30811007046d68634894a7c2723c5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:49.630387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:49.643640 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be468b79-311b-4a5b-8c22-45fa0f2a571e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:49.649171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:49.649840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:49.650117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:49.650389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] PATH_INFO: `/v3/users/62e8a7e7dba64398b0f6bdbc51c1e495` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:49.651560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e8a7e7dba64398b0f6bdbc51c1e495)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:49.653533 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:49.679749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de30811007046d68634894a7c2723c5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.680355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de30811007046d68634894a7c2723c5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.688388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de30811007046d68634894a7c2723c5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.696788 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de30811007046d68634894a7c2723c5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.697137 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de30811007046d68634894a7c2723c5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.706147 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e8a7e7dba64398b0f6bdbc51c1e495'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.711497 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:49.714096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e8a7e7dba64398b0f6bdbc51c1e495'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.715566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e8a7e7dba64398b0f6bdbc51c1e495'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.716361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e8a7e7dba64398b0f6bdbc51c1e495'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.718854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1335/2655] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:49 2022] DELETE /identity/v3/users/2de30811007046d68634894a7c2723c5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:49.728769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:49.730215 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e8a7e7dba64398b0f6bdbc51c1e495'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:49.745998 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1321/2656] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:05:49 2022] DELETE /identity/v3/users/62e8a7e7dba64398b0f6bdbc51c1e495 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:50.248395 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.249871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.333521 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.335370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.346814 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98292181-2bc0-4126-8e96-027eb1d55e3b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.408931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.410202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.410468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.410755 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] PATH_INFO: `/v3/projects/29a9462a18d24287b396b06231028d3a` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.411958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a9462a18d24287b396b06231028d3a)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.413907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.418382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ded64f-7000-4379-9774-310401980852 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.423028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.423937 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.424218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.424526 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] PATH_INFO: `/v3/projects/0b662a9a55d14dbe93a13e982e157186` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.425759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b662a9a55d14dbe93a13e982e157186)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.427606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.464798 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a9462a18d24287b396b06231028d3a', 'request_id': 'req-903022ed-e19d-4d6b-904b-37452d0b3c8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.473339 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a9462a18d24287b396b06231028d3a', 'request_id': 'req-903022ed-e19d-4d6b-904b-37452d0b3c8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.478628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b662a9a55d14dbe93a13e982e157186', 'request_id': 'req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.478943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a9462a18d24287b396b06231028d3a', 'request_id': 'req-903022ed-e19d-4d6b-904b-37452d0b3c8f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.487478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b662a9a55d14dbe93a13e982e157186', 'request_id': 'req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.491274 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] The token cache is being invalidate because project 29a9462a18d24287b396b06231028d3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:50.491568 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.492210 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-903022ed-e19d-4d6b-904b-37452d0b3c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.493671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b662a9a55d14dbe93a13e982e157186', 'request_id': 'req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.494305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1336/2657] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:50 2022] DELETE /identity/v3/projects/29a9462a18d24287b396b06231028d3a => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:50.500737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd33c00-784a-4d83-8e18-adf8e7fa7dc2 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.501022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd33c00-784a-4d83-8e18-adf8e7fa7dc2 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.501301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd33c00-784a-4d83-8e18-adf8e7fa7dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.505900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] The token cache is being invalidate because project 0b662a9a55d14dbe93a13e982e157186 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:05:50.506195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.506836 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2a0cc3c-0eb0-46da-99bb-1207611eb4b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:05:50.508361 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1322/2658] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:05:50 2022] DELETE /identity/v3/projects/0b662a9a55d14dbe93a13e982e157186 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:50.513258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18432142-3b09-462d-8e2d-092142bc66b8 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.513664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18432142-3b09-462d-8e2d-092142bc66b8 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.514047 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18432142-3b09-462d-8e2d-092142bc66b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.524881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-edd33c00-784a-4d83-8e18-adf8e7fa7dc2 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:50.526218 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-edd33c00-784a-4d83-8e18-adf8e7fa7dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.534205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-18432142-3b09-462d-8e2d-092142bc66b8 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:50.535950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-18432142-3b09-462d-8e2d-092142bc66b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.560080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:50.593885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1337/2659] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:50.599874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1323/2660] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:50.601214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.601511 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.601791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.606003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.606302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.606616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.626177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:50.626762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:50.627370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.629064 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:50.636008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1324/2661] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:50.640252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.643847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1338/2662] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:50.645327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.650724 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aad8a79-12aa-4a97-8141-972a8bd67e78 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.655622 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.656246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.656562 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.656720 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de696625-8af8-412e-be3c-a86c9ec2ee54 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.656826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.657512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:50.657946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.659670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.662949 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.663826 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.664244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.664690 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.665840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:05:50.666224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.666929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:50.667809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.677130 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1325/2663] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 21:05:50.677737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.687743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1339/2664] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:05:50.689403 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee71b77-8660-4f2b-8a44-90f9ceb84d4c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.689539 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.694868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.695480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.695771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.696058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.696823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.698890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.700202 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78a9a-b525-4a81-b99b-0e8e32c6c8c4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.704891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.705500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.705789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.706068 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.707561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.709505 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.730053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1326/2665] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/projects => generated 407 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:50.734695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.738963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1340/2666] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:50.742955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.802583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42d627-419c-434b-8973-339f959c2155 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.807556 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f23fd6-bcd0-4ac5-b17b-dac8d50b24c4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.807802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.808461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.808751 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.809056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.809671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.811586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.812314 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.812969 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.813243 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.813520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.814181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.816872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.839686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1327/2667] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/users => generated 304 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:50.843553 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1341/2668] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:05:50.844141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.844629 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.854210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145782c2-b367-49f8-8b5c-ac5cd5b85f4c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.854908 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df254bb-502f-4882-b9cb-01f6c173cd2f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.859049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743220d-e017-4424-be43-04094012de8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.859653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743220d-e017-4424-be43-04094012de8f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.859825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.859983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743220d-e017-4424-be43-04094012de8f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.860189 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743220d-e017-4424-be43-04094012de8f admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.860531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.860816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.860975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743220d-e017-4424-be43-04094012de8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.861081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.861760 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.863094 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743220d-e017-4424-be43-04094012de8f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.864406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.873489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1328/2669] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:50.876437 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1342/2670] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:50.877656 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743220d-e017-4424-be43-04094012de8f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.879772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.888594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743220d-e017-4424-be43-04094012de8f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.889320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04190eb-a7e4-425b-bf3b-ac9104bf9a4b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.893245 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.893860 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.894154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.894285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.894548 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.894838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.895127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.895236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.895416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.896168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.896771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.897789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.907886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1329/2671] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:50.912805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:50.923984 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09305569-bc3a-411c-b97f-d997a4ec8e59 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:50.930630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:50.931332 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.931611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.931889 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] PATH_INFO: `/v3/projects/4e0b81efdf7c44048e3502b9dda2077c/users/bbe787e31683433a9b6edb3aebb92f5f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:50.949355 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e0b81efdf7c44048e3502b9dda2077c, user_id=bbe787e31683433a9b6edb3aebb92f5f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:50.952107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668bf69-9626-472d-a932-685486ddbb18 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:50.968979 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1330/2672] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:50 2022] PUT /identity/v3/projects/4e0b81efdf7c44048e3502b9dda2077c/users/bbe787e31683433a9b6edb3aebb92f5f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:50.981066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:50.981312 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:50.981476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.011382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] MFA Rules not processed for user `bbe787e31683433a9b6edb3aebb92f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:51.012664 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:51.047429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1343/2673] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:51.052722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.062936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae077d-02f3-4ccf-9dfc-c8938026e3c6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.084726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1331/2674] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:50 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:51.089490 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.100027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:51.122822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.123508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.123776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.124054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] PATH_INFO: `/v3/projects/d142eaffafa84ba89b53b2e8ab43639f/users/b9319c0e1d7d4ced94b0817272b29bb4/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.127122 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:51.141135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d142eaffafa84ba89b53b2e8ab43639f, user_id=b9319c0e1d7d4ced94b0817272b29bb4, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.144483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119f6530-fac3-466a-8a8d-1ef0c5682747 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.156485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631e2101-961e-4fe4-b409-e1ef30ab5a69 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.158203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1344/2675] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:51 2022] PUT /identity/v3/projects/d142eaffafa84ba89b53b2e8ab43639f/users/b9319c0e1d7d4ced94b0817272b29bb4/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:05:51.164455 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.164747 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.165024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.183508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:51.184879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:51.213482 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.214104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.214380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.214650 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.215300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.217654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.247946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1332/2676] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:05:51 2022] POST /identity/v3/projects => generated 407 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:51.248175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1345/2677] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:05:51.250966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.257059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.258442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:51.322154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae461c-62bc-4c8a-82c7-f7a49eb0e642 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.322328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8542340-6ddf-427d-9cde-4775b5585e86 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.327362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.328154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.328509 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.328809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.329589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.332322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.355162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1333/2678] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:05:51 2022] POST /identity/v3/users => generated 310 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:05:51.359352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.369448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21b8e4c-d1b1-4d30-88c8-beb9d5f57fc9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.374228 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.374900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.375248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.375590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.375714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.376457 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.376564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.376749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.377026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.379728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.386879 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.388306 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.390073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1334/2679] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:05:51.394710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.398898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1346/2680] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:05:51.405538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e6833c-d9a6-44eb-85ca-02cdd7ecf91d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.411638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.412311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.412625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.412928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] PATH_INFO: `/v3/projects/7c6668c422c14bb492a4d428f8d8aaac/users/6107d43e41c84e37b91bc86fddef45a2/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.428965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6668c422c14bb492a4d428f8d8aaac, user_id=6107d43e41c84e37b91bc86fddef45a2, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.431634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.435694 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:51.448480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1335/2681] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:51 2022] PUT /identity/v3/projects/7c6668c422c14bb492a4d428f8d8aaac/users/6107d43e41c84e37b91bc86fddef45a2/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:51.452958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.463878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804126c1-b18c-4d60-b191-6ea7ba3ab768 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.510158 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:51.527934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.528685 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.529034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.529728 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.529993 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.531677 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf86553-b251-43e4-9228-8260d27c4c36 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.543625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1347/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:05:51.548155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.570402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1febd09-64be-4530-8549-4998a58fb419 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.579360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.580101 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.580422 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.580702 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] PATH_INFO: `/v3/projects/7c6668c422c14bb492a4d428f8d8aaac/users/6107d43e41c84e37b91bc86fddef45a2/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.582938 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6668c422c14bb492a4d428f8d8aaac, user_id=6107d43e41c84e37b91bc86fddef45a2, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.585678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.598125 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1336/2683] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:05:51 2022] PUT /identity/v3/projects/7c6668c422c14bb492a4d428f8d8aaac/users/6107d43e41c84e37b91bc86fddef45a2/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:05:51.609898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.610419 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.610524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.639546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] MFA Rules not processed for user `6107d43e41c84e37b91bc86fddef45a2`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:51.640679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:51.715450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1348/2684] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:51 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:51.726974 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:51.729128 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:51.799250 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64637376-a877-4a05-8bc5-b50870ba4bc5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:51.804472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:51.805100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:51.805383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:51.805654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:51.815857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:51.817480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2024bf-e53e-4391-8ce7-7c5c189fe185 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:51.827186 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1337/2685] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:51.995418 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:52.005980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d3cd9f-7c44-4997-854b-3e1cb70b5536 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:52.009925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:52.010623 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:52.010929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:52.011204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:52.058136 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52af6-bc51-4a5f-a246-bdb937fefd1a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:52.058561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52af6-bc51-4a5f-a246-bdb937fefd1a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:52.059270 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db52af6-bc51-4a5f-a246-bdb937fefd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:52.091172 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-2db52af6-bc51-4a5f-a246-bdb937fefd1a None None] MFA Rules not processed for user `b9319c0e1d7d4ced94b0817272b29bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:52.092539 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2db52af6-bc51-4a5f-a246-bdb937fefd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:52.095861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:52.098428 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:52.111208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1349/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:51 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:05:52.162590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1338/2687] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:52.173811 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:52.175627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:52.187491 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b48331-46c8-4777-8922-92839532bd97 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:52.191817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:52.192743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:52.193022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:52.193290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:52.203230 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:52.204891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:52.215248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1350/2688] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:54.674758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:54.675823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:54.675823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:54.699046 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] MFA Rules not processed for user `b9319c0e1d7d4ced94b0817272b29bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:05:54.700302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:54.713056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1339/2689] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:05:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:05:54.982694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:54.984057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:55.041006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc413268-8966-4b84-a240-85817eac0b86 service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:55.044849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:55.045467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:55.045750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:55.046022 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:55.047766 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:55.054443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:55.058885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:55.060294 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:55.069268 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1351/2690] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:05:55.396158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:05:55.397838 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:55.469763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2614a-9ff3-47d8-8268-29484f960cae None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:05:55.475018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:05:55.475645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:05:55.475932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:05:55.476209 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:05:55.478004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:05:55.490445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:05:55.492016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:05:55.496304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:05:55.504303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1340/2691] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:05:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:01.249327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:01.250877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:01.325516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68797a5a-0d7f-487b-9691-3803d94180c3 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:01.331203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:01.331922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:01.332224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:01.332531 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:01.334259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:01.346365 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:01.348598 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ad2ca3-94ae-4f66-ae34-54eddc808496 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:01.359014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1352/2692] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:05.505415 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:06.871701 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:06.934326 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65135d16-3a2a-4097-acb4-6317bd3dbb70 service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:06.938777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:06.939401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:06.939680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:06.939953 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] PATH_INFO: `/v3/users/bbe787e31683433a9b6edb3aebb92f5f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:06.941811 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe787e31683433a9b6edb3aebb92f5f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:06.943801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:06.993162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe787e31683433a9b6edb3aebb92f5f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.001062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe787e31683433a9b6edb3aebb92f5f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.001691 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe787e31683433a9b6edb3aebb92f5f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.002304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe787e31683433a9b6edb3aebb92f5f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.015005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe787e31683433a9b6edb3aebb92f5f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.028160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1341/2693] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:06 2022] DELETE /identity/v3/users/bbe787e31683433a9b6edb3aebb92f5f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:07.032619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:07.033121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:07.033232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:07.052599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:07.053819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:07.115120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1353/2694] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:07.125360 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:07.126952 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:07.189629 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74bfc5-a814-4d10-bad6-391e4400d1d1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:07.194096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:07.194727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:07.195012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:07.195290 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:07.206950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:07.208437 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:07.219462 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1342/2695] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:07.501852 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:07.504023 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:07.527141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20b0a3-2b78-4d24-bfa6-cf0e8b4ceedb None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:07.531866 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:07.532833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:07.533242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:07.533579 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] PATH_INFO: `/v3/projects/4e0b81efdf7c44048e3502b9dda2077c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:07.535174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e0b81efdf7c44048e3502b9dda2077c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:07.537282 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:07.588214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e0b81efdf7c44048e3502b9dda2077c', 'request_id': 'req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.595952 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e0b81efdf7c44048e3502b9dda2077c', 'request_id': 'req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.602082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e0b81efdf7c44048e3502b9dda2077c', 'request_id': 'req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.614886 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] The token cache is being invalidate because project 4e0b81efdf7c44048e3502b9dda2077c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:07.615172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.615828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.617388 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1354/2696] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:07 2022] DELETE /identity/v3/projects/4e0b81efdf7c44048e3502b9dda2077c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:07.621392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:07.631826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07e294-e9eb-4d4f-9f7e-96f375685deb service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:07.654844 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:07.701707 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:07.702348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:07.702631 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:07.702914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] PATH_INFO: `/v3/users/6107d43e41c84e37b91bc86fddef45a2` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:07.704057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6107d43e41c84e37b91bc86fddef45a2)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:07.706076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:07.754912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6107d43e41c84e37b91bc86fddef45a2'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.762412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6107d43e41c84e37b91bc86fddef45a2'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.763081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6107d43e41c84e37b91bc86fddef45a2'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.763636 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6107d43e41c84e37b91bc86fddef45a2'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.776826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-edd91a33-3d14-4ace-8cfb-c2a568503977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6107d43e41c84e37b91bc86fddef45a2'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:07.791269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1343/2697] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:07 2022] DELETE /identity/v3/users/6107d43e41c84e37b91bc86fddef45a2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:08.093223 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.094672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:08.103671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:08.110793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb84ad-bc18-4b5d-80a5-dfc589822ae5 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.168351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.169001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.169289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.169564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] PATH_INFO: `/v3/projects/7c6668c422c14bb492a4d428f8d8aaac` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.170697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6668c422c14bb492a4d428f8d8aaac)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.172630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.218600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6668c422c14bb492a4d428f8d8aaac', 'request_id': 'req-952b9756-b4b3-42bd-88d2-c1de36c0dd07'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:08.226312 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6668c422c14bb492a4d428f8d8aaac', 'request_id': 'req-952b9756-b4b3-42bd-88d2-c1de36c0dd07'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:08.231962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6668c422c14bb492a4d428f8d8aaac', 'request_id': 'req-952b9756-b4b3-42bd-88d2-c1de36c0dd07'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:08.243429 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] The token cache is being invalidate because project 7c6668c422c14bb492a4d428f8d8aaac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:08.243715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:08.244380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-952b9756-b4b3-42bd-88d2-c1de36c0dd07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:08.246482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1355/2698] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:08 2022] DELETE /identity/v3/projects/7c6668c422c14bb492a4d428f8d8aaac => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:08.251913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.252404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.252506 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.272862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:08.274071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:08.338748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1344/2699] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:08.346822 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.347180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.347493 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.367558 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:08.368790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:08.377252 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1356/2700] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:08.383049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.397673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5051d-8f6a-41d7-842e-3efa9a775d2d None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.404497 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.405394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.405927 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.406334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.407413 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:08.407881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.410376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.429061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1345/2701] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:08.431143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.442195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60914e4f-0000-42af-9a0b-f794c0850f76 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.446946 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.447563 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.447844 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.448118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.448813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.450694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.481734 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1357/2702] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:08 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:08.486183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.498404 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60439ff-94ad-4c52-a5c7-b160d3be8ed1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.546315 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:08.584630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.585442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.585832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.586236 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.587025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.589127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.613273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1346/2703] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:08 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:08.617644 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.628461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51163c53-3701-4b27-8a2f-ca040a7401d5 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.633332 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.634029 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.634377 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.634718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.635478 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.637192 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.651433 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1358/2704] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:08.652790 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.664119 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c0edb-79be-420b-a5f3-c4ddb3d48864 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.669583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.670274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.670623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.670963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.671730 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.673696 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d62f201-926b-460b-ba61-3e87128096bd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.684892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1347/2705] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:08.689144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.699591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8990d4-ab5e-4aea-b12e-01cbfc45dbce admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.704187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.704880 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.705163 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.705436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] PATH_INFO: `/v3/projects/d700f161ff034e9e974113371bb585cc/users/e41a93ab5e554b88a9f5f079ffb8bd6f/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.720860 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d700f161ff034e9e974113371bb585cc, user_id=e41a93ab5e554b88a9f5f079ffb8bd6f, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.724710 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.740379 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1359/2706] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:08 2022] PUT /identity/v3/projects/d700f161ff034e9e974113371bb585cc/users/e41a93ab5e554b88a9f5f079ffb8bd6f/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:08.752302 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.753222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.753835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.783379 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] MFA Rules not processed for user `e41a93ab5e554b88a9f5f079ffb8bd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:08.785292 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:08.854378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1348/2707] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:06:08.866465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:08.868441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:08.904333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:08.953219 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c22d90-8ef7-4162-87c2-7d05ad845402 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:08.956870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f260187ace2043849cb3a4ab780899a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:08.957486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:08.957997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:08.958135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:08.966672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:08.968101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1cbc8-42a9-4bf9-aac5-96a359fb22ab service glance] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:08.980106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1360/2708] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:10.132907 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:10.134294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.200737 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3295f-d029-4384-8e54-a72306a60b75 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:10.205765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:10.207223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:10.207223 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:10.207623 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] PATH_INFO: `/v3/users/1a5ba3081b2048b4b703e7016786a517` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:10.208817 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5ba3081b2048b4b703e7016786a517)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:10.210796 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:10.254476 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:10.261821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5ba3081b2048b4b703e7016786a517'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.270608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5ba3081b2048b4b703e7016786a517'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.271257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5ba3081b2048b4b703e7016786a517'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.271867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5ba3081b2048b4b703e7016786a517'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.284006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5ba3081b2048b4b703e7016786a517'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.297239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1349/2709] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:10 2022] DELETE /identity/v3/users/1a5ba3081b2048b4b703e7016786a517 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:10.304884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:10.305371 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:10.305498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:10.328393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:10.329957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.391854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1361/2710] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:10.401671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:10.403071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.464071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b49ca6-c87a-4b5b-b303-27ffde0bc4c1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:10.467678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:10.468296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:10.468621 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:10.468900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:10.478569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:10.480610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:10.499685 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1350/2711] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:10.672926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:10.676094 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.782981 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97428ec7-b1ce-4e09-9694-8175e0f78549 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:10.795193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:10.796257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:10.796790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:10.797226 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:10.799790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.812411 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:10.813989 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:10.816442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:10.818221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:10.832401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1362/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:10.838965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7867aab7-ad56-4652-b115-ed2ac7a067ac service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:10.848906 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:10.849793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:10.850171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:10.850560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] PATH_INFO: `/v3/projects/12af94af1dd34026ada996f394f15efd` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:10.851966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12af94af1dd34026ada996f394f15efd)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:10.854336 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:10.911992 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12af94af1dd34026ada996f394f15efd', 'request_id': 'req-977f5bbc-133a-4b85-a765-2ee15452a040'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.922527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12af94af1dd34026ada996f394f15efd', 'request_id': 'req-977f5bbc-133a-4b85-a765-2ee15452a040'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.929269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12af94af1dd34026ada996f394f15efd', 'request_id': 'req-977f5bbc-133a-4b85-a765-2ee15452a040'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.943038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] The token cache is being invalidate because project 12af94af1dd34026ada996f394f15efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:10.943427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.944162 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:10.945973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1351/2713] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:10 2022] DELETE /identity/v3/projects/12af94af1dd34026ada996f394f15efd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:10.949854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:10.961809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf8074e-32e4-40ed-9eab-0acf9d63bdcc service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:10.999521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:11.026134 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.026709 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.026988 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.027473 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] PATH_INFO: `/v3/users/3bd170057fde45b1931a5d4f8df4aa24` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.028796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd170057fde45b1931a5d4f8df4aa24)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.030898 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.078528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd170057fde45b1931a5d4f8df4aa24'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.079278 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd170057fde45b1931a5d4f8df4aa24'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.090681 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd170057fde45b1931a5d4f8df4aa24'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.100521 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd170057fde45b1931a5d4f8df4aa24'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.101279 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-fb805375-a794-4238-af0d-d9b0de2ff7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd170057fde45b1931a5d4f8df4aa24'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.120080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1363/2714] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:10 2022] DELETE /identity/v3/users/3bd170057fde45b1931a5d4f8df4aa24 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:11.413543 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.415041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:11.428391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5bbc-133a-4b85-a765-2ee15452a040 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.472831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:11.491426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.492078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.492388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.492713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] PATH_INFO: `/v3/projects/a52493491a9f494e9eaa6a5cd327292c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.493913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a52493491a9f494e9eaa6a5cd327292c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.495829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.558596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a52493491a9f494e9eaa6a5cd327292c', 'request_id': 'req-293b273b-bd44-4d49-a144-4d5d830b5035'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.567171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a52493491a9f494e9eaa6a5cd327292c', 'request_id': 'req-293b273b-bd44-4d49-a144-4d5d830b5035'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.572886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a52493491a9f494e9eaa6a5cd327292c', 'request_id': 'req-293b273b-bd44-4d49-a144-4d5d830b5035'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.584886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] The token cache is being invalidate because project a52493491a9f494e9eaa6a5cd327292c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:11.585102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.585778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-293b273b-bd44-4d49-a144-4d5d830b5035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:11.587407 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1352/2715] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:11 2022] DELETE /identity/v3/projects/a52493491a9f494e9eaa6a5cd327292c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:11.592303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.592780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.592877 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.612910 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:11.614130 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:11.673905 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1364/2716] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:11.681929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.682221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.682502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.702625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:11.703875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:11.712883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1353/2717] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:11.717450 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.728875 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076faa3d-f4fe-4436-b6cf-6942b64b7325 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.733544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.734159 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.734442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.734716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.735414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:11.735717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.737813 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.749471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1365/2718] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:11.754594 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.764002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b24570-7b88-4b89-845e-2b7d2b90806c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.769920 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.770680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.770960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.771231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.772010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.774378 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.807717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1354/2719] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:11 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:11.809184 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.819717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdac885-b55a-4b8c-8e13-fe74c25660c7 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.861883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:11.881712 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.882334 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.882617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.882892 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.883572 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.885498 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.906824 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1366/2720] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:11 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:11.911120 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.921439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c160084-6346-4cf3-a2d8-a2369d02a8d1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.926924 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.927576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.927893 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.928163 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.928928 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.930667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.942062 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1355/2721] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:11.945925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.956024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb32-fa3a-43d4-8255-d18b53f07dc6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:11.960744 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:11.961345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:11.961627 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:11.961904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:11.962587 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:11.964176 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca13790-c012-495c-8d81-94e1985a7440 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:11.975423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1367/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:11.981382 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:11.992978 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfde52-55ba-4dfc-8a28-3e027e5f389e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.000026 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.001283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.001576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.002076 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] PATH_INFO: `/v3/projects/0c2550adae0645f5b7c40978d3bafade/users/8989878b6ac846b785a9a537e5d7d5a8/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.026996 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c2550adae0645f5b7c40978d3bafade, user_id=8989878b6ac846b785a9a537e5d7d5a8, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.031856 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.052295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1356/2723] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:11 2022] PUT /identity/v3/projects/0c2550adae0645f5b7c40978d3bafade/users/8989878b6ac846b785a9a537e5d7d5a8/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:12.063857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.064150 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.064461 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.095872 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] MFA Rules not processed for user `8989878b6ac846b785a9a537e5d7d5a8`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:12.097213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:12.165212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1368/2724] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:12.169840 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.236785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e00d0-270c-4e3b-bdd9-68a562861c2f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.242527 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.243149 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.243434 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.243710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.244412 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.246356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.260450 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:12.276549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1357/2725] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:12.280794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.292948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea67e2-5e7b-46ab-b3a0-55fac0123d3b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.352318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.352958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.353242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.353517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.354171 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.356080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.378483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1369/2726] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:12.382602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.393266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5440c-205e-4639-b9bc-9ccbf5523a1b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.398100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.398717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.399002 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.399281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.399970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.401611 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.413341 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1358/2727] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:12.417561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.430806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964a5a9-52d1-4a21-8887-37199bd8f182 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.436099 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.436761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.437042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.437318 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.438008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.439610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d232e1-e9d6-499f-8786-f0ec8b25482f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.449815 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1370/2728] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:12.454102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.464983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd963b4-c91b-4027-8bba-c704fb033f1f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.469741 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.470406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.470710 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.470975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] PATH_INFO: `/v3/projects/eed2bcdd1c4e4eccac878c6f2d2229d6/users/e808f2905daf46c497eda230aa62304b/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.488508 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eed2bcdd1c4e4eccac878c6f2d2229d6, user_id=e808f2905daf46c497eda230aa62304b, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.491173 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.504427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1359/2729] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:12 2022] PUT /identity/v3/projects/eed2bcdd1c4e4eccac878c6f2d2229d6/users/e808f2905daf46c497eda230aa62304b/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:12.519445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.519746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.520154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.558761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] MFA Rules not processed for user `e808f2905daf46c497eda230aa62304b`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:12.559980 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:12.584670 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:12.640390 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1371/2730] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:12.641523 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.712107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81792-25fc-42c6-89f7-3f6c6c7769f1 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.717088 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.717708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.717986 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.718259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.718916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.720866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.750639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1360/2731] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:12.751882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.762807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197ae01d-b6f6-44d5-99e0-6793957c5d34 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.820658 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.821271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.821555 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.821839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.822488 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.824413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.847923 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1372/2732] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:12 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:12.851910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.863311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d817ddd5-2dee-4b20-a377-327a6735de4e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.868259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.869050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.869365 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.869713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.870417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.872279 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.886065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1361/2733] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:12.890281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.901615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef531c7-ee32-4699-92c1-3d42c341734f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:12.906528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:12.907273 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:12.907564 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:12.907840 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] PATH_INFO: `/v3/projects/f977d9dffeba41f094c782cf088b8e8c/users/c143af2c931f4a2bae4e639d1ba49f68/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:12.925290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f977d9dffeba41f094c782cf088b8e8c, user_id=c143af2c931f4a2bae4e639d1ba49f68, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:12.927922 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:12.942635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1373/2734] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:12 2022] PUT /identity/v3/projects/f977d9dffeba41f094c782cf088b8e8c/users/c143af2c931f4a2bae4e639d1ba49f68/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:12.946910 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:12.958471 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758cd21-38f4-42fb-99af-748471f3e494 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:13.019035 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:13.020295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:13.020610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:13.020891 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:13.021598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:13.023239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f78e3-35a3-4609-9c4f-eb638764403c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:13.037205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1362/2735] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:13.038344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:13.051034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ce656-e4ea-4b19-9609-f7f6757ba175 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:13.058208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:13.058208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:13.058533 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:13.058633 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] PATH_INFO: `/v3/projects/f977d9dffeba41f094c782cf088b8e8c/users/c143af2c931f4a2bae4e639d1ba49f68/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:13.060686 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f977d9dffeba41f094c782cf088b8e8c, user_id=c143af2c931f4a2bae4e639d1ba49f68, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:13.063417 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:13.075247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1374/2736] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:13 2022] PUT /identity/v3/projects/f977d9dffeba41f094c782cf088b8e8c/users/c143af2c931f4a2bae4e639d1ba49f68/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:13.089217 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:13.089609 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:13.089901 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:13.120929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] MFA Rules not processed for user `c143af2c931f4a2bae4e639d1ba49f68`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:13.122143 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:13.139405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:13.195494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1363/2737] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:13.207244 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:13.208972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:13.275177 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c4903-b0f0-467d-9ed3-29fcbd5e7e9e admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:13.278756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:13.279373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:13.279656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:13.279932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:13.289992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:13.291423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:13.303436 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1375/2738] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:13.930804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:13.940269 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c873a161-209a-4c10-b80f-ac737fa2e96f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:13.943866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:13.944483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:13.944761 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:13.945032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:14.001489 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:14.003037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:14.011695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1364/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:18.084032 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:18.093280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cb341-cf5c-43ad-9abe-d24b5483a4da service cinder] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:18.097065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:18.097682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:18.097964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:18.098238 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:18.111091 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:18.163501 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:18.164995 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da6fd4-bea9-42e4-9b72-1f47293b9e57 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:18.177793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1376/2740] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:20.346174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:20.411477 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5b38-b3d5-4569-b79c-081c93cdc572 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:20.416100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:20.416769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:20.417071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:20.417343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] PATH_INFO: `/v3/users/8989878b6ac846b785a9a537e5d7d5a8` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:20.418485 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8989878b6ac846b785a9a537e5d7d5a8)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:20.420457 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:20.470463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8989878b6ac846b785a9a537e5d7d5a8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:20.479398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8989878b6ac846b785a9a537e5d7d5a8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:20.480014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8989878b6ac846b785a9a537e5d7d5a8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:20.480660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8989878b6ac846b785a9a537e5d7d5a8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:20.481900 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:20.496472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8989878b6ac846b785a9a537e5d7d5a8'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:20.511308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1365/2741] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:20 2022] DELETE /identity/v3/users/8989878b6ac846b785a9a537e5d7d5a8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:20.526385 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:20.526941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:20.526941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:20.548926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:20.551778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:20.626643 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1377/2742] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:20.637059 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:20.638486 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:20.706855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b10ac1-09dd-4705-b858-a669ebdcab7c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:20.710536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:20.711155 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:20.711440 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:20.711715 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:20.723905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:20.725080 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:20.735328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1366/2743] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:21.039221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:21.040745 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:21.054133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46ae7b2-5267-43a7-b3b2-693669a9c7b1 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:21.059867 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:21.060596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.060883 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.061153 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] PATH_INFO: `/v3/projects/0c2550adae0645f5b7c40978d3bafade` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.062410 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c2550adae0645f5b7c40978d3bafade)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:21.064359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:21.123661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c2550adae0645f5b7c40978d3bafade', 'request_id': 'req-dd7252a6-fde8-443b-9b59-7213f901e24f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.133490 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c2550adae0645f5b7c40978d3bafade', 'request_id': 'req-dd7252a6-fde8-443b-9b59-7213f901e24f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.140611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c2550adae0645f5b7c40978d3bafade', 'request_id': 'req-dd7252a6-fde8-443b-9b59-7213f901e24f'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.152079 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] The token cache is being invalidate because project 0c2550adae0645f5b7c40978d3bafade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:21.152468 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.153084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.154583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1378/2744] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:21 2022] DELETE /identity/v3/projects/0c2550adae0645f5b7c40978d3bafade => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:21.158652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:21.168793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e89cbf-b5ec-40e9-8af1-93d36f73b926 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:21.236063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:21.236752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.237743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.238064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] PATH_INFO: `/v3/users/e808f2905daf46c497eda230aa62304b` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.239259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e808f2905daf46c497eda230aa62304b)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:21.241212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:21.291322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e808f2905daf46c497eda230aa62304b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.300869 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e808f2905daf46c497eda230aa62304b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.301552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e808f2905daf46c497eda230aa62304b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.302444 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e808f2905daf46c497eda230aa62304b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.319616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e808f2905daf46c497eda230aa62304b'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.324651 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:21.336553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1367/2745] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:21 2022] DELETE /identity/v3/users/e808f2905daf46c497eda230aa62304b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:21.630616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:21.632110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:21.647921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7252a6-fde8-443b-9b59-7213f901e24f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:21.685936 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:21.710583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:21.711157 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.711665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.711964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] PATH_INFO: `/v3/projects/eed2bcdd1c4e4eccac878c6f2d2229d6` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.713193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eed2bcdd1c4e4eccac878c6f2d2229d6)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:21.715476 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:21.762700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eed2bcdd1c4e4eccac878c6f2d2229d6', 'request_id': 'req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.771792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eed2bcdd1c4e4eccac878c6f2d2229d6', 'request_id': 'req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.778124 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eed2bcdd1c4e4eccac878c6f2d2229d6', 'request_id': 'req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.790485 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] The token cache is being invalidate because project eed2bcdd1c4e4eccac878c6f2d2229d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:21.790769 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.791569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.793508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1379/2746] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:21 2022] DELETE /identity/v3/projects/eed2bcdd1c4e4eccac878c6f2d2229d6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:21.797988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:21.799866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:21.812999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10f92a-3bdd-49cd-93e9-449c43890b48 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:21.872318 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:21.873009 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:21.873295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:21.873574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] PATH_INFO: `/v3/users/c143af2c931f4a2bae4e639d1ba49f68` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:21.875216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c143af2c931f4a2bae4e639d1ba49f68)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:21.878053 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:21.927708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c143af2c931f4a2bae4e639d1ba49f68'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.936473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c143af2c931f4a2bae4e639d1ba49f68'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.937169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c143af2c931f4a2bae4e639d1ba49f68'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.937850 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c143af2c931f4a2bae4e639d1ba49f68'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.951653 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-221b1022-a7a0-42f6-b27a-47d018b85d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c143af2c931f4a2bae4e639d1ba49f68'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:21.965881 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1368/2747] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:21 2022] DELETE /identity/v3/users/c143af2c931f4a2bae4e639d1ba49f68 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:22.258967 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.261607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:22.279240 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a00d0b-c2da-40da-bed1-e97d4b3b63f3 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.357881 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.358543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.358827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.359104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] PATH_INFO: `/v3/projects/f977d9dffeba41f094c782cf088b8e8c` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.360267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f977d9dffeba41f094c782cf088b8e8c)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.362809 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.411645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f977d9dffeba41f094c782cf088b8e8c', 'request_id': 'req-5b157121-2de1-4710-937a-54b2269a106c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:22.419603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f977d9dffeba41f094c782cf088b8e8c', 'request_id': 'req-5b157121-2de1-4710-937a-54b2269a106c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:22.426384 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f977d9dffeba41f094c782cf088b8e8c', 'request_id': 'req-5b157121-2de1-4710-937a-54b2269a106c'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:22.433927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.439342 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] The token cache is being invalidate because project f977d9dffeba41f094c782cf088b8e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:22.439821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:22.442082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5b157121-2de1-4710-937a-54b2269a106c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:22.443789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1380/2748] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:22 2022] DELETE /identity/v3/projects/f977d9dffeba41f094c782cf088b8e8c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:22.449654 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.449942 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.450220 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.472683 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:22.474003 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:22.535232 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1369/2749] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:22.543926 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.544221 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.544524 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.567494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:22.568767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:22.576764 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1381/2750] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:22.581199 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.591469 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc0b98-20af-4d2c-85b9-45886ce4fdbf None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.596321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.597146 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.597426 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.597699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.598391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:22.598690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.600301 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.610131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:22.614823 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1370/2751] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:22.619254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.629849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d770a-8406-4ab3-befd-6cb75a185b76 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.634512 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.635121 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.635399 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.635671 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.636363 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.638254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.678669 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1382/2752] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:22 2022] POST /identity/v3/projects => generated 391 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:22.680023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.690252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fff85-c083-4d4c-aa0e-a263637d0405 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.749965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.750596 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.750888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.751171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.751870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.754433 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.776688 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1371/2753] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:22 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:22.781120 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.792362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c965b-7cbf-48fa-9493-1427c9777aa8 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.797945 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.799821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.799821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.800084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.800754 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.802458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.813164 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1383/2754] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:22.817765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.827828 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aafb998-bfe6-4196-bb49-89c05229bbb0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.833061 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.833675 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.833960 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.834241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.834930 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.836660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb90a4e-c614-4185-aed5-61326626bfec admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.849249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1372/2755] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:22.853786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:22.864035 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bf85b-d67b-40ab-b4b2-c3d7f2b3797b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:22.868687 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:22.869349 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.869630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.869903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] PATH_INFO: `/v3/projects/3a2265c27659486e9457a4ad966db43e/users/605d8330ed8d4894b58ebaec7146e58d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.884920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2265c27659486e9457a4ad966db43e, user_id=605d8330ed8d4894b58ebaec7146e58d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:22.887653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:22.902610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1384/2756] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:22 2022] PUT /identity/v3/projects/3a2265c27659486e9457a4ad966db43e/users/605d8330ed8d4894b58ebaec7146e58d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:22.915311 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:22.915591 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:22.915863 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:22.945791 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] MFA Rules not processed for user `605d8330ed8d4894b58ebaec7146e58d`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:22.947001 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:23.016252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1373/2757] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:23.022237 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.090958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ada4e4-8beb-42a2-9556-e57f3148914b admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.095595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.096855 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.097767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.098047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.098723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.100680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.129151 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1385/2758] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:23 2022] POST /identity/v3/projects => generated 391 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:23.133261 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.144826 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca583054-ec3d-4d4a-858c-ed2175283eda None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.202212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.202845 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.203127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.203401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.204063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.206552 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.230633 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1374/2759] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:23 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:23.231154 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.234847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.246253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94fbec-b249-4ce9-8792-79cf95e0c2bd admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.251257 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.251893 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.252180 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.252483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.253508 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.255024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.269716 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1386/2760] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:23.273792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.284553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc08f1-2356-49b7-9cc8-9fafae474427 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.289784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.290467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.290746 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.291019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] PATH_INFO: `/v3/projects/195b5f79b2344795a2f6f6ab9afd6514/users/35ec338590bf4e77be4caff1ddb4f713/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.306955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195b5f79b2344795a2f6f6ab9afd6514, user_id=35ec338590bf4e77be4caff1ddb4f713, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.309704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.322771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1375/2761] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:23 2022] PUT /identity/v3/projects/195b5f79b2344795a2f6f6ab9afd6514/users/35ec338590bf4e77be4caff1ddb4f713/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:23.328409 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.341260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2d8d3-bce1-437e-9d7a-8c44c1677586 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.399927 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.400594 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.400870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.401149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.401833 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.403416 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b97b48-9118-4058-8417-b8dd8a196482 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.414141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1387/2762] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:23.418409 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.429660 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3619f5d-60c3-4214-a0f0-b3531a9a40a7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.434399 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.435100 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.435385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.435793 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] PATH_INFO: `/v3/projects/195b5f79b2344795a2f6f6ab9afd6514/users/35ec338590bf4e77be4caff1ddb4f713/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.437914 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=195b5f79b2344795a2f6f6ab9afd6514, user_id=35ec338590bf4e77be4caff1ddb4f713, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.440785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.456602 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1376/2763] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:23 2022] PUT /identity/v3/projects/195b5f79b2344795a2f6f6ab9afd6514/users/35ec338590bf4e77be4caff1ddb4f713/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:23.467471 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.467786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.468066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.477181 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:23.510138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] MFA Rules not processed for user `35ec338590bf4e77be4caff1ddb4f713`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:23.511381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:23.587232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1388/2764] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:23.598325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.600044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:23.659757 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa11e59-10f3-4d37-8b4e-279ad8b96b5d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.663418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.664038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.664317 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.664620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.674680 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.676116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.685748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1377/2765] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:23.738625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:23.740008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:23.751947 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d3e77-4633-41c8-9d73-0e51046be57a None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:23.756904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:23.757516 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:23.757797 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:23.758070 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:23.816024 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:23.817522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca64f52-3f5d-4635-92a1-629b75d545a0 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:23.828964 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1389/2766] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:28.389093 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:28.466962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8adda66-b301-420b-a13c-7d17bd0356eb service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:28.472253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:28.472940 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:28.473219 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:28.473494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] PATH_INFO: `/v3/users/605d8330ed8d4894b58ebaec7146e58d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:28.474665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=605d8330ed8d4894b58ebaec7146e58d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:28.476604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:28.530628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605d8330ed8d4894b58ebaec7146e58d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:28.539251 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605d8330ed8d4894b58ebaec7146e58d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:28.539905 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605d8330ed8d4894b58ebaec7146e58d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:28.540585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605d8330ed8d4894b58ebaec7146e58d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:28.552532 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:28.557815 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605d8330ed8d4894b58ebaec7146e58d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:28.571346 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1378/2767] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:28 2022] DELETE /identity/v3/users/605d8330ed8d4894b58ebaec7146e58d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:28.575731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:28.576236 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:28.576383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:28.595630 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:28.597452 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:28.671108 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1390/2768] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:28.681445 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:28.682866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:28.741617 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e92f8-d61c-4e4a-b88d-df898d6b9632 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:28.745224 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:28.745835 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:28.746116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:28.746387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:28.755943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:28.757395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:28.766645 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1379/2769] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:29.046542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.048286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:29.060635 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bd972f-9413-4458-bf13-00f67d903631 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:29.065792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:29.066440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.066719 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.066992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] PATH_INFO: `/v3/projects/3a2265c27659486e9457a4ad966db43e` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.068155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a2265c27659486e9457a4ad966db43e)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:29.070135 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:29.099972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:29.124971 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a2265c27659486e9457a4ad966db43e', 'request_id': 'req-053360f3-3baa-465b-ad39-cefb949c7502'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.133612 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a2265c27659486e9457a4ad966db43e', 'request_id': 'req-053360f3-3baa-465b-ad39-cefb949c7502'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.139247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a2265c27659486e9457a4ad966db43e', 'request_id': 'req-053360f3-3baa-465b-ad39-cefb949c7502'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.150362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] The token cache is being invalidate because project 3a2265c27659486e9457a4ad966db43e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:29.150642 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.151259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.152804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1391/2770] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:29 2022] DELETE /identity/v3/projects/3a2265c27659486e9457a4ad966db43e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:29.156806 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.166866 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef0fb0-d2e8-46f2-9b84-4c38da51372b service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:29.229950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:29.229950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.229950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.229950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] PATH_INFO: `/v3/users/35ec338590bf4e77be4caff1ddb4f713` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.230878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ec338590bf4e77be4caff1ddb4f713)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:29.232081 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:29.280025 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ec338590bf4e77be4caff1ddb4f713'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.289174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ec338590bf4e77be4caff1ddb4f713'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.289779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ec338590bf4e77be4caff1ddb4f713'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.290388 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ec338590bf4e77be4caff1ddb4f713'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.302878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-13ae7cab-7b21-4a56-a7ef-a7dfa4087e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ec338590bf4e77be4caff1ddb4f713'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.317398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1380/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:29 2022] DELETE /identity/v3/users/35ec338590bf4e77be4caff1ddb4f713 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:29.597693 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.599178 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:29.615348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053360f3-3baa-465b-ad39-cefb949c7502 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:29.677315 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:29.678082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.678358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.678625 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] PATH_INFO: `/v3/projects/195b5f79b2344795a2f6f6ab9afd6514` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.679757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] RBAC: Authorizing `identity:delete_project(project_id=195b5f79b2344795a2f6f6ab9afd6514)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:29.681676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:29.731394 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195b5f79b2344795a2f6f6ab9afd6514', 'request_id': 'req-b2e585a3-232a-4bba-a207-e24bc9d0661b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.739843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195b5f79b2344795a2f6f6ab9afd6514', 'request_id': 'req-b2e585a3-232a-4bba-a207-e24bc9d0661b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.745443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195b5f79b2344795a2f6f6ab9afd6514', 'request_id': 'req-b2e585a3-232a-4bba-a207-e24bc9d0661b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.756941 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] The token cache is being invalidate because project 195b5f79b2344795a2f6f6ab9afd6514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:29.757222 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.757847 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-b2e585a3-232a-4bba-a207-e24bc9d0661b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:29.759373 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1392/2772] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:29 2022] DELETE /identity/v3/projects/195b5f79b2344795a2f6f6ab9afd6514 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:29.764174 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.764706 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.764816 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.773677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:29.785921 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:29.787188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:29.847063 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1381/2773] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:29.855038 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.855324 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.855595 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.875692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:29.876904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:29.884825 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1393/2774] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:29.889098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.899227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d3b1af-fad0-44e1-82d7-0020d6a79b45 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:29.903862 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:29.904496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.904780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.905050 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.905731 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:29.906032 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:29.907616 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:29.920018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1382/2775] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:29.924743 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.934071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37626fd1-3f13-483b-a5fa-f815dd9899b5 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:29.938689 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:29.939297 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:29.939578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:29.940039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:29.940903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:29.942795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:29.970653 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1394/2776] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:29 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:29.975576 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:29.985432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a6de3-3f31-4bdf-a0f9-16a21b1b5021 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.040837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.041364 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.041643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.041915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.042572 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.044496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.065160 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1383/2777] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:29 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:30.069065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.079127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc0a09-818f-4d41-b368-d04e9a64a394 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.084749 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.085348 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.085628 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.085932 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.086615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.088201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.099168 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1395/2778] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:30.102999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.113067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6521deb-2226-4800-bbfc-888e7ed54f0e admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.117606 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.118205 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.118483 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.118755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.119432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.121042 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc086e-1365-4927-9ee4-0aef730c0dbd admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.131762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1384/2779] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:30.135730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.140790 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.149933 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb2da5-3ee8-44d5-bf5a-9aa86c5382d9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.154943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.155614 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.155897 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.156169 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] PATH_INFO: `/v3/projects/8c2c4e8628204f8d98d65119537604de/users/0af2bb21f2374aaabd752fc11f7bdf1d/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.171772 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c2c4e8628204f8d98d65119537604de, user_id=0af2bb21f2374aaabd752fc11f7bdf1d, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.175082 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.192740 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1396/2780] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:30 2022] PUT /identity/v3/projects/8c2c4e8628204f8d98d65119537604de/users/0af2bb21f2374aaabd752fc11f7bdf1d/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:30.265231 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.265496 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.265593 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.295106 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] MFA Rules not processed for user `0af2bb21f2374aaabd752fc11f7bdf1d`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:30.296239 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:30.367417 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.371558 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1385/2781] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:30.376014 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.438704 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589a7bb-c118-4440-affb-42b7cac32e32 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.443818 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.444569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.444856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.445133 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.445789 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.447692 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.474267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1397/2782] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:30 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:30.478507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.489711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705e293-cf5d-40e3-bac1-594fa0deba17 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.551977 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.552499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.552783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.553058 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.553740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.555644 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.577323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1386/2783] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:30 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:30.583198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.602784 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9f303-0458-45f8-9820-e5d3fd9a52b6 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.614123 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.614750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.615033 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.615305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.616036 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.617711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.630447 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1398/2784] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:30.634492 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.646316 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2f92aa-21ca-495b-b7e3-13db346e49fb admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.652098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.653041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.653595 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.653886 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] PATH_INFO: `/v3/projects/249d7b27b16e48b99a106a2f7c431cc5/users/b35084ea84064c1ba21794eee5dd2295/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.672182 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249d7b27b16e48b99a106a2f7c431cc5, user_id=b35084ea84064c1ba21794eee5dd2295, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.675902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.689442 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1387/2785] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:30 2022] PUT /identity/v3/projects/249d7b27b16e48b99a106a2f7c431cc5/users/b35084ea84064c1ba21794eee5dd2295/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:30.694136 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.704484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22461b9f-2cbb-417e-8149-7af9c384dedf admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.770610 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.771251 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.771522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.771937 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.772673 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.774269 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f5db4-04f4-42de-b83b-363eb0cd6a5b admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.786095 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1399/2786] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:30.789965 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.801470 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569dcfdd-3817-454c-9c44-a352b46576b5 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:30.806608 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:30.807249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.807529 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.807988 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] PATH_INFO: `/v3/projects/249d7b27b16e48b99a106a2f7c431cc5/users/b35084ea84064c1ba21794eee5dd2295/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.809865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=249d7b27b16e48b99a106a2f7c431cc5, user_id=b35084ea84064c1ba21794eee5dd2295, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:30.812779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:30.824728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1388/2787] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:30 2022] PUT /identity/v3/projects/249d7b27b16e48b99a106a2f7c431cc5/users/b35084ea84064c1ba21794eee5dd2295/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:30.835701 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:30.835992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:30.836267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:30.863943 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] MFA Rules not processed for user `b35084ea84064c1ba21794eee5dd2295`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:30.865189 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:30.902265 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:30.934884 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1400/2788] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:30.946377 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:30.948041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:31.006936 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c21dbab-387a-4ba8-a3c8-59b842ae8aa0 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.010585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.011195 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.011472 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.011744 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.020777 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.022193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.031393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1389/2789] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:31.136323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:31.146389 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245c38f-555e-407c-8c07-6bb63f61c7ec None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:31.150040 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:31.150646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:31.150930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:31.151204 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:31.211333 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:31.212806 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393fa936-eaa1-45cf-80e1-72b6b93c40ae service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:31.222638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1401/2790] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:34.863956 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:34.933518 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:34.939266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36011be-a6fc-4408-9cfb-a4a9525d064d service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:34.945545 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:34.946198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:34.946480 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:34.946758 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] PATH_INFO: `/v3/users/0af2bb21f2374aaabd752fc11f7bdf1d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:34.947925 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af2bb21f2374aaabd752fc11f7bdf1d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:34.949882 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:34.996478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af2bb21f2374aaabd752fc11f7bdf1d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.004323 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af2bb21f2374aaabd752fc11f7bdf1d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.005019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af2bb21f2374aaabd752fc11f7bdf1d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.005632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af2bb21f2374aaabd752fc11f7bdf1d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.017620 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af2bb21f2374aaabd752fc11f7bdf1d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.029763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1390/2791] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:34 2022] DELETE /identity/v3/users/0af2bb21f2374aaabd752fc11f7bdf1d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:35.034986 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.035486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.035600 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.054863 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:35.056062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:35.117228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1402/2792] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:35.127010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.128830 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:35.187630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7cc1e-041f-4e94-976e-cf60a5fb6f6c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.191349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.191964 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.192248 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.192557 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.201618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.203051 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.213613 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1391/2793] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:35.485812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.487438 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:35.499584 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe70a7-671e-4087-9a9d-463819637f47 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.504994 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.505638 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.505917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.506190 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] PATH_INFO: `/v3/projects/8c2c4e8628204f8d98d65119537604de` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.507343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c2c4e8628204f8d98d65119537604de)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.509304 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.563067 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c2c4e8628204f8d98d65119537604de', 'request_id': 'req-759c6ef4-e15c-41cc-8784-623a7350d385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.570714 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c2c4e8628204f8d98d65119537604de', 'request_id': 'req-759c6ef4-e15c-41cc-8784-623a7350d385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.576196 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c2c4e8628204f8d98d65119537604de', 'request_id': 'req-759c6ef4-e15c-41cc-8784-623a7350d385'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.578058 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.591566 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] The token cache is being invalidate because project 8c2c4e8628204f8d98d65119537604de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:35.591977 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.592757 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.594253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1403/2794] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:35 2022] DELETE /identity/v3/projects/8c2c4e8628204f8d98d65119537604de => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:35.598755 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:35.608197 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e953228-719a-4913-8a52-5dfdb3e8ac41 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:35.672478 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:35.673286 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:35.673570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:35.673871 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] PATH_INFO: `/v3/users/b35084ea84064c1ba21794eee5dd2295` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:35.675183 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35084ea84064c1ba21794eee5dd2295)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:35.677142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:35.697775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:35.724904 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35084ea84064c1ba21794eee5dd2295'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.731919 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35084ea84064c1ba21794eee5dd2295'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.732538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35084ea84064c1ba21794eee5dd2295'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.733131 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35084ea84064c1ba21794eee5dd2295'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.745672 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b9a5e065-1002-4737-8441-f7a739203bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35084ea84064c1ba21794eee5dd2295'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:35.758555 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1392/2795] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:35 2022] DELETE /identity/v3/users/b35084ea84064c1ba21794eee5dd2295 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:36.045934 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.047660 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:36.062071 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6ef4-e15c-41cc-8784-623a7350d385 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.125161 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.125804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.126081 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.126353 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] PATH_INFO: `/v3/projects/249d7b27b16e48b99a106a2f7c431cc5` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.128195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=249d7b27b16e48b99a106a2f7c431cc5)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.130106 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.176096 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '249d7b27b16e48b99a106a2f7c431cc5', 'request_id': 'req-5012621c-94b1-4ff9-8947-051a73c50e92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:36.183401 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '249d7b27b16e48b99a106a2f7c431cc5', 'request_id': 'req-5012621c-94b1-4ff9-8947-051a73c50e92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:36.188963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '249d7b27b16e48b99a106a2f7c431cc5', 'request_id': 'req-5012621c-94b1-4ff9-8947-051a73c50e92'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:36.201683 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] The token cache is being invalidate because project 249d7b27b16e48b99a106a2f7c431cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:36.201973 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:36.202656 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-5012621c-94b1-4ff9-8947-051a73c50e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:36.204224 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1404/2796] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:36 2022] DELETE /identity/v3/projects/249d7b27b16e48b99a106a2f7c431cc5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:36.209144 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.209610 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.209733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.229725 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:36.230949 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:36.289340 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1393/2797] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:36.297162 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.297449 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.297731 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.318841 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:36.320057 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:36.328102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1405/2798] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:36.333689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.344011 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27da1e3-5c7b-49c7-b792-9766975fc78c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.349267 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.349888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.350169 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.350441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.351157 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:36.351456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.353103 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.365792 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1394/2799] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:36.370271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.379793 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65c32-2d60-4ec7-9f53-9af669278c8b None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.384443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.385303 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.385430 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.385672 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.386345 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.388235 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.416779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1406/2800] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:36.421164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.433014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fb4d3-e913-40be-95db-97df7c7e08d9 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.491687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.492352 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.492873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.493156 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.493883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.495808 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.517060 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1395/2801] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:36.521962 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.532523 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdda494-7bc8-471d-9e74-3207d47cb4b2 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.537940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.538550 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.538846 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.539101 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.539785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.541408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.552979 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1407/2802] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:36.557092 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.567342 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798aa71-2840-4c0f-a641-f4e5309b3632 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.571837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.572513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.572859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.573123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.573929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.575637 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d2920-e0d0-4387-a572-a29f8b61ffd3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.586739 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1396/2803] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:36.590911 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.601175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f5c7ce-a4df-4045-b837-f2af97fd744f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.605985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.606637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.606915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.607206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] PATH_INFO: `/v3/projects/4a736999039e46ccbca44baf65f4e72d/users/f6a3352fd6f747d58c2222c22f10ee2b/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.621838 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a736999039e46ccbca44baf65f4e72d, user_id=f6a3352fd6f747d58c2222c22f10ee2b, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.624483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.638199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1408/2804] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:36 2022] PUT /identity/v3/projects/4a736999039e46ccbca44baf65f4e72d/users/f6a3352fd6f747d58c2222c22f10ee2b/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:36.651868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.652257 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.652630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.661392 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:36.685127 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] MFA Rules not processed for user `f6a3352fd6f747d58c2222c22f10ee2b`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:36.686287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:36.755304 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1397/2805] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:36.760069 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.787785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:36.825904 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121593c3-6aad-439d-b337-8119dbbcb689 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.831413 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.832054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.832355 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.832645 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.833319 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.835232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.861722 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1409/2806] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:36.865990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.876781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e361228-b3c8-4e98-8d1c-f67a9ef7c2a1 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.935569 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.936096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.936396 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.936671 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.937331 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.939234 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.963708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1398/2807] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:36 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:36.965154 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:36.975679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662714e-9757-4fc2-8b2a-21dba09fc204 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:36.980378 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:36.981008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:36.981290 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:36.981561 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:36.982242 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:36.983821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:36.994960 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1410/2808] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:37.010363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.010363 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69751880-f9fa-41d6-b155-509cfc96dff3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.015778 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.016513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.016798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.017073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.018337 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.019962 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b5a0fc-a329-4c5d-810c-4210d7f21ea5 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.035167 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1399/2809] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:37.040361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.065267 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0de3c-7b8f-442d-bc4b-080a1c58a0fb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.070724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.071129 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.071406 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.071680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] PATH_INFO: `/v3/projects/14f8ad7c886d45f1b1bcd7c9fa1a4513/users/1259b0428a784689ab41414f0fd116e8/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.092586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f8ad7c886d45f1b1bcd7c9fa1a4513, user_id=1259b0428a784689ab41414f0fd116e8, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.092791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.112537 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1411/2810] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:37 2022] PUT /identity/v3/projects/14f8ad7c886d45f1b1bcd7c9fa1a4513/users/1259b0428a784689ab41414f0fd116e8/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:37.123216 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.123686 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.124027 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.163618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] MFA Rules not processed for user `1259b0428a784689ab41414f0fd116e8`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:37.165006 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:37.231717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:37.240544 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1400/2811] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:37.242407 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.320321 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c78a92-926b-45b4-8cef-7f9afc514f49 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.325232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.326002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.326331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.326661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.327369 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.329320 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.362445 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1412/2812] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:37 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:37.363963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.375598 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd598fa6-283e-4dfd-9c3f-dcd4835d2fe6 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.437207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.437883 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.438116 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.438395 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.439056 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.441193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.463310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1401/2813] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:37 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:37.467421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.478387 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267906c5-0ed6-4c18-9c86-e832b5cdab88 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.482924 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.483520 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.483795 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.484062 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.484778 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.486383 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.496796 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1413/2814] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:37.501089 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.511090 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b5098-fc03-43d6-adaf-76557e243c7c admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.515652 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.516435 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.516726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.517015 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] PATH_INFO: `/v3/projects/c2e410956fe849cb8d09225a1c802b5c/users/e2f1f559ff8e4a9ba2331d9d32d2cb43/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.533057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e410956fe849cb8d09225a1c802b5c, user_id=e2f1f559ff8e4a9ba2331d9d32d2cb43, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.537096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.551903 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1402/2815] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:37 2022] PUT /identity/v3/projects/c2e410956fe849cb8d09225a1c802b5c/users/e2f1f559ff8e4a9ba2331d9d32d2cb43/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:37.556012 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.566607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239349b0-03ef-4d16-a891-0daedc1b9edf admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.620203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:37.636206 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.636856 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.637140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.637415 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.638104 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.639694 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320148c-c0ca-4b0b-af7c-ffd306af1fd8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.650119 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1414/2816] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:37.654349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.666225 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbbd04-ca53-4a6a-a1e9-e6e8db16c22f admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.671619 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.672431 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.672752 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.673343 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] PATH_INFO: `/v3/projects/c2e410956fe849cb8d09225a1c802b5c/users/e2f1f559ff8e4a9ba2331d9d32d2cb43/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.676112 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e410956fe849cb8d09225a1c802b5c, user_id=e2f1f559ff8e4a9ba2331d9d32d2cb43, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.678892 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.689722 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1403/2817] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:37 2022] PUT /identity/v3/projects/c2e410956fe849cb8d09225a1c802b5c/users/e2f1f559ff8e4a9ba2331d9d32d2cb43/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:37.701054 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.701344 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.701619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.730756 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] MFA Rules not processed for user `e2f1f559ff8e4a9ba2331d9d32d2cb43`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:37.731965 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:37.801270 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1415/2818] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:37.812868 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.815678 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:37.872667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b469a38d-bdd4-467d-bb70-50e698c3f565 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.876240 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.877421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.877711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.878204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:37.886943 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:37.888387 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:37.899780 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1404/2819] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:37.943575 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:37.944931 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:37.958292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca06ef-35b4-4073-ad21-4c99297e9ae0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:37.961918 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:37.962528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:37.962812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:37.963085 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.020264 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:38.021721 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b0374-ea5f-48f1-bd8a-e44b2c315309 service cinder] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:38.030983 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1416/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:38.438873 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:38.441018 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:38.500465 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:38.507405 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bedd6d-fc62-43bf-9850-31c0144d4c0c service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:38.512348 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:38.513038 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.513325 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.513605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] PATH_INFO: `/v3/users/e41a93ab5e554b88a9f5f079ffb8bd6f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.514870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e41a93ab5e554b88a9f5f079ffb8bd6f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:38.516857 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:38.565566 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e41a93ab5e554b88a9f5f079ffb8bd6f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:38.572870 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e41a93ab5e554b88a9f5f079ffb8bd6f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:38.573488 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e41a93ab5e554b88a9f5f079ffb8bd6f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:38.574107 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e41a93ab5e554b88a9f5f079ffb8bd6f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:38.586282 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e41a93ab5e554b88a9f5f079ffb8bd6f'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:38.599102 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1405/2821] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:38 2022] DELETE /identity/v3/users/e41a93ab5e554b88a9f5f079ffb8bd6f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:38.603906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.604489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.604606 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.624191 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:38.625459 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:38.685100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1417/2822] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:38.695187 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:38.696625 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:38.758848 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedf281-31f3-463d-a8e6-28f7bee2d786 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:38.762400 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:38.763019 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:38.763300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:38.763577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:38.772999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:38.774449 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5453be-8c04-41d9-bf9b-f9cf520eff96 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:38.784827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1406/2823] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:39.056227 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.057684 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:39.070065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe30e2-a3a3-43db-96f6-38e470efa4ac None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.074608 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.075247 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.075528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.075792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] PATH_INFO: `/v3/projects/d700f161ff034e9e974113371bb585cc` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.076957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d700f161ff034e9e974113371bb585cc)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.078839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.084381 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=100 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.131083 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd700f161ff034e9e974113371bb585cc', 'request_id': 'req-52a60eae-d54c-431a-8266-c3913cb87973'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:39.138723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd700f161ff034e9e974113371bb585cc', 'request_id': 'req-52a60eae-d54c-431a-8266-c3913cb87973'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:39.145396 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd700f161ff034e9e974113371bb585cc', 'request_id': 'req-52a60eae-d54c-431a-8266-c3913cb87973'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:39.157546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] The token cache is being invalidate because project d700f161ff034e9e974113371bb585cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:39.157836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:39.159496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-52a60eae-d54c-431a-8266-c3913cb87973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:39.161061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1418/2824] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:39 2022] DELETE /identity/v3/projects/d700f161ff034e9e974113371bb585cc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:39.166429 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.166726 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.167007 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.186957 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:39.188148 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:39.249689 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1407/2825] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:39.257919 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.258212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.258502 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.278307 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:39.279494 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:39.287496 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1419/2826] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:39.291865 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.302303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95a5e8-d0d1-4fba-ba27-2935608e7653 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.308991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.308991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.308991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.308991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.309834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86872) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:39.309834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.310638 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.323393 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1408/2827] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:39.329817 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.340902 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c3811-c9a0-446b-8fdf-c4646815ceed None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.345545 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.346260 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.346528 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.346804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.347467 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.349408 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.376688 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1420/2828] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:39.381950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.388277 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.392276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5d4ae-ea99-4392-baaf-748427f631d8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.454561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.455723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.456528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.456804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.457479 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.459398 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.480493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1409/2829] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:39.484591 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.502655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb5bac-745e-4020-b2f5-a02e3ec70eed admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.507534 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.508148 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.508469 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.508752 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.509446 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.511118 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.522578 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1421/2830] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:39.526574 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.537104 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f7072-083e-474b-9168-4ceb92e217f3 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.541750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.542351 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.542630 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.542899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.543628 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.545259 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9d6bd-a57a-4286-be1e-709755d95e9a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.556141 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1410/2831] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:39.560117 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.570479 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822a42c-71cc-4193-9894-42cb085c5aa0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.575093 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.575739 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] REQUEST_METHOD: `PUT` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.576021 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.576293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] PATH_INFO: `/v3/projects/34ce72f3b88e443c8f89a48d79173743/users/c05cc85e66fe4bce916ba2addcfad395/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.591812 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ce72f3b88e443c8f89a48d79173743, user_id=c05cc85e66fe4bce916ba2addcfad395, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.594465 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.607915 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1422/2832] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:39 2022] PUT /identity/v3/projects/34ce72f3b88e443c8f89a48d79173743/users/c05cc85e66fe4bce916ba2addcfad395/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:39.618728 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.619012 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.619285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.647005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] MFA Rules not processed for user `c05cc85e66fe4bce916ba2addcfad395`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:39.648507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:39.715945 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1411/2833] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:39.720913 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.787819 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6dd7d4-885b-459e-ac2f-d5a45e045b1f admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.792680 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.793357 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.793640 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.793920 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.794646 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.796617 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.824305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1423/2834] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:39.829519 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.842273 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f33d68-35fd-4a9a-997c-733b7d1eb12f None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.899809 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.900513 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.900797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.901075 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] PATH_INFO: `/v3/users` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.901742 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.903705 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.926222 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1412/2835] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:39 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:06:39.930272 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.941138 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58024340-234e-4d66-a42a-01d3939a9fff admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.945776 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.946393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.946676 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.946949 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:39.947782 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:39.952374 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:39.959773 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.965832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1424/2836] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:39.971420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:39.977037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:39.986252 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2bd8fc-67e3-4a89-8a36-5b3458775b62 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:39.995074 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:39.995946 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:39.996243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:39.996564 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] PATH_INFO: `/v3/projects/5e6143406ca34702967cff6df0dee384/users/b6e2ea8f1b914820b4f6f943801fddef/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.012632 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6143406ca34702967cff6df0dee384, user_id=b6e2ea8f1b914820b4f6f943801fddef, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.015303 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.029615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1413/2837] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:39 2022] PUT /identity/v3/projects/5e6143406ca34702967cff6df0dee384/users/b6e2ea8f1b914820b4f6f943801fddef/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:40.035302 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.052737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9d9ef-e1ee-4234-8637-2b3f9f6a88d9 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.144330 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.145375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.145853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.146325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.147549 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.150504 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c644aae-d298-41ff-921f-cee7b55ab6ed admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.168930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1425/2838] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:06:40.173293 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.184902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ceec1-1fda-4ffe-90d2-c8def29fed4d admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.189958 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.190762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.191041 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.191319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] PATH_INFO: `/v3/projects/5e6143406ca34702967cff6df0dee384/users/b6e2ea8f1b914820b4f6f943801fddef/roles/8b2191da1ad1452b8f5f5353376e06bb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.193534 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6143406ca34702967cff6df0dee384, user_id=b6e2ea8f1b914820b4f6f943801fddef, role_id=8b2191da1ad1452b8f5f5353376e06bb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.196266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.207902 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1414/2839] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:40 2022] PUT /identity/v3/projects/5e6143406ca34702967cff6df0dee384/users/b6e2ea8f1b914820b4f6f943801fddef/roles/8b2191da1ad1452b8f5f5353376e06bb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:40.276174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.276507 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.276792 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.307604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] MFA Rules not processed for user `b6e2ea8f1b914820b4f6f943801fddef`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:40.308857 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:40.384874 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1426/2840] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:40.398321 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:40.400401 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:40.466468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f809195-da01-45d6-b5b0-eee6aca242f6 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:40.470253 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:40.471727 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:40.472014 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:40.472358 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:40.538570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:40.540017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c304985-fe02-4cb4-91b4-3071c499f5fe service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:40.551068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1415/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:06:42.018697 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:42.020484 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:42.087301 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7fa7-778a-432a-b75f-2e688fff17d8 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:42.091878 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:42.092621 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.092906 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.093185 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] PATH_INFO: `/v3/users/f6a3352fd6f747d58c2222c22f10ee2b` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.094492 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6a3352fd6f747d58c2222c22f10ee2b)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:42.097343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:42.120140 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:42.149213 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6a3352fd6f747d58c2222c22f10ee2b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.149807 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6a3352fd6f747d58c2222c22f10ee2b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.157392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6a3352fd6f747d58c2222c22f10ee2b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.166317 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6a3352fd6f747d58c2222c22f10ee2b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.167001 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6a3352fd6f747d58c2222c22f10ee2b'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.184885 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1427/2842] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:42 2022] DELETE /identity/v3/users/f6a3352fd6f747d58c2222c22f10ee2b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:42.191677 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.192188 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.192714 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.223670 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:42.225775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:42.297335 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1416/2843] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:42.308187 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:42.309647 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:42.376482 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78adcf-c3f9-424e-a0c1-00d236279526 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:42.380887 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:42.381500 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.381785 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.382059 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.391000 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:42.393112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:42.402475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1428/2844] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:42.677448 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:42.678932 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:42.692783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d9b0c-de5a-409b-973f-adc67438df0a None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:42.697374 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:42.698021 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.698300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.698583 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] PATH_INFO: `/v3/projects/4a736999039e46ccbca44baf65f4e72d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.699708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a736999039e46ccbca44baf65f4e72d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:42.701665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:42.718349 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:42.751711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a736999039e46ccbca44baf65f4e72d', 'request_id': 'req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.760212 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a736999039e46ccbca44baf65f4e72d', 'request_id': 'req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.765955 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a736999039e46ccbca44baf65f4e72d', 'request_id': 'req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.776909 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] The token cache is being invalidate because project 4a736999039e46ccbca44baf65f4e72d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:42.777196 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.777878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.779425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1417/2845] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:42 2022] DELETE /identity/v3/projects/4a736999039e46ccbca44baf65f4e72d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:42.783616 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:42.793631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1255e12-2d32-477a-a372-a4ebcf9f82d9 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:42.851228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:42.851859 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:42.852139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:42.852443 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] PATH_INFO: `/v3/users/1259b0428a784689ab41414f0fd116e8` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:42.853574 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1259b0428a784689ab41414f0fd116e8)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:42.855466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:42.899583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:42.901453 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1259b0428a784689ab41414f0fd116e8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.902182 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1259b0428a784689ab41414f0fd116e8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.911607 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1259b0428a784689ab41414f0fd116e8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.922198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1259b0428a784689ab41414f0fd116e8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.922780 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1259b0428a784689ab41414f0fd116e8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:42.941352 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1429/2846] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:42 2022] DELETE /identity/v3/users/1259b0428a784689ab41414f0fd116e8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:43.252391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.253890 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:43.268764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5e126-856d-49aa-bc5b-09c6bdb37f39 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.337859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.338547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.338831 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.339105 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] PATH_INFO: `/v3/projects/14f8ad7c886d45f1b1bcd7c9fa1a4513` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.340294 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f8ad7c886d45f1b1bcd7c9fa1a4513)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.342271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.390769 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f8ad7c886d45f1b1bcd7c9fa1a4513', 'request_id': 'req-1f36e93f-b85a-4274-bef9-4d3ec28d7028'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.398913 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f8ad7c886d45f1b1bcd7c9fa1a4513', 'request_id': 'req-1f36e93f-b85a-4274-bef9-4d3ec28d7028'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.405226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f8ad7c886d45f1b1bcd7c9fa1a4513', 'request_id': 'req-1f36e93f-b85a-4274-bef9-4d3ec28d7028'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.416804 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] The token cache is being invalidate because project 14f8ad7c886d45f1b1bcd7c9fa1a4513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:43.417097 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.417775 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.419300 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1418/2847] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:43 2022] DELETE /identity/v3/projects/14f8ad7c886d45f1b1bcd7c9fa1a4513 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:43.426328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.430483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:43.445700 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68782188-dbde-47dd-b6b9-efa180073b32 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.505854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.506434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.506718 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.506992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] PATH_INFO: `/v3/users/e2f1f559ff8e4a9ba2331d9d32d2cb43` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.508142 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f1f559ff8e4a9ba2331d9d32d2cb43)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.510112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:43.558882 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f1f559ff8e4a9ba2331d9d32d2cb43'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.559536 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f1f559ff8e4a9ba2331d9d32d2cb43'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.567562 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f1f559ff8e4a9ba2331d9d32d2cb43'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.574836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f1f559ff8e4a9ba2331d9d32d2cb43'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.575460 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-16550f21-3751-49db-8fe0-88d9e08877a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f1f559ff8e4a9ba2331d9d32d2cb43'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:43.592049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1430/2848] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:43 2022] DELETE /identity/v3/users/e2f1f559ff8e4a9ba2331d9d32d2cb43 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:43.879381 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:43.880950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:43.896114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36e93f-b85a-4274-bef9-4d3ec28d7028 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:43.932499 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:43.960849 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:43.961502 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:43.961783 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:43.962085 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] PATH_INFO: `/v3/projects/c2e410956fe849cb8d09225a1c802b5c` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:43.963283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e410956fe849cb8d09225a1c802b5c)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:43.965241 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.013044 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e410956fe849cb8d09225a1c802b5c', 'request_id': 'req-060baaf2-dfef-4e42-b071-a393419d9e8a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.020763 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e410956fe849cb8d09225a1c802b5c', 'request_id': 'req-060baaf2-dfef-4e42-b071-a393419d9e8a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.026695 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e410956fe849cb8d09225a1c802b5c', 'request_id': 'req-060baaf2-dfef-4e42-b071-a393419d9e8a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.038980 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] The token cache is being invalidate because project c2e410956fe849cb8d09225a1c802b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:44.039258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.039899 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-060baaf2-dfef-4e42-b071-a393419d9e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.041454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1419/2849] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:43 2022] DELETE /identity/v3/projects/c2e410956fe849cb8d09225a1c802b5c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:44.047343 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.047637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.048203 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.070529 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:44.071724 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.111289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.135486 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1431/2850] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:44.145271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.145560 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.145875 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.166494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:44.167740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.176295 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1420/2851] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:44.180767 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.190985 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa9646-976d-4dc7-b3ed-628e0e9ae9dd None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.196143 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.196801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.197080 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.197354 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.198047 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:06:44.198351 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.199957 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.210733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.212207 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.219955 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1432/2852] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:06:44.222515 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.228171 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d457-0a42-4734-8d72-cb00a7be9b8c None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.232546 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2094c8c-25cb-4a10-af84-9805b38175a0 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.237221 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.237441 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.237849 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.238049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.238155 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.238334 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.238543 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] PATH_INFO: `/v3/projects` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.238643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] PATH_INFO: `/v3/users/c05cc85e66fe4bce916ba2addcfad395` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.239013 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.239878 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05cc85e66fe4bce916ba2addcfad395)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.240984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.241923 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.269930 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1433/2853] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:44.274440 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.300895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05cc85e66fe4bce916ba2addcfad395'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.309281 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05cc85e66fe4bce916ba2addcfad395'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.310064 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05cc85e66fe4bce916ba2addcfad395'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.310700 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05cc85e66fe4bce916ba2addcfad395'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.323867 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-b4e309a7-98bb-483d-bd1d-873801f2e57e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05cc85e66fe4bce916ba2addcfad395'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.337394 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1421/2854] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:44 2022] DELETE /identity/v3/users/c05cc85e66fe4bce916ba2addcfad395 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:44.341704 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.342016 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.342280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.347286 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd642a-65f8-4e30-9ef5-6f6f60c34917 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.364029 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:44.365514 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.391720 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.407705 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.408542 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.408832 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.409110 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.409803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.411765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.435439 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1422/2855] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:06:44.440662 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1434/2856] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:06:44.441406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.442859 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.448414 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.450810 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.455894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927778f-735b-4a8d-9c7e-2216c301b373 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.461129 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.462052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.462535 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.463052 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.463759 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.465697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.478589 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1423/2857] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:44.482425 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.493762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a8995-ce1e-465a-8f8c-d3ba4fd8a760 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.504655 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:44.510801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.511516 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.511647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.512180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.512767 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.514391 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.525894 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9c063-e416-436d-8728-fc177426eba2 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.531092 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.531779 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.532052 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.532323 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.532603 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1424/2858] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 21:06:44.536764 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.542804 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.544207 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e9a6c-62d6-4601-8f3a-afa1a4eb2526 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.546561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0131b8-13be-4553-927d-3d027fe0e813 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.552461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.553142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.553430 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.553748 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] PATH_INFO: `/v3/projects/41991cdb5b2e455086e3c3699462671f/users/010c2c107d26433b8be2e5a9db969f3a/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.555002 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1435/2859] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:44.571468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41991cdb5b2e455086e3c3699462671f, user_id=010c2c107d26433b8be2e5a9db969f3a, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.574328 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.590242 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1425/2860] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:06:44 2022] PUT /identity/v3/projects/41991cdb5b2e455086e3c3699462671f/users/010c2c107d26433b8be2e5a9db969f3a/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:44.603172 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.603431 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.603761 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.637821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] MFA Rules not processed for user `010c2c107d26433b8be2e5a9db969f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:44.639325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.718284 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1436/2861] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:44.731966 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.734180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.796779 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c5c2-e426-4a07-b086-b58c3a203558 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.802040 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.802703 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.802987 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.803271 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.812983 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.815432 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.824717 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1426/2862] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:06:44.856175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:44.857765 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:44.928637 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc518-cc2e-4d00-89c2-33cabe34c574 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:44.933231 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:44.933870 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:44.934147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:44.934425 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] PATH_INFO: `/v3/projects/34ce72f3b88e443c8f89a48d79173743` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:44.935604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ce72f3b88e443c8f89a48d79173743)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:44.937514 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:44.988843 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ce72f3b88e443c8f89a48d79173743', 'request_id': 'req-8e2f5360-12af-470f-97b0-2949c24c874a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:44.997248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ce72f3b88e443c8f89a48d79173743', 'request_id': 'req-8e2f5360-12af-470f-97b0-2949c24c874a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.004098 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ce72f3b88e443c8f89a48d79173743', 'request_id': 'req-8e2f5360-12af-470f-97b0-2949c24c874a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.015896 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] The token cache is being invalidate because project 34ce72f3b88e443c8f89a48d79173743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:45.016214 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.017063 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.018682 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1437/2863] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:44 2022] DELETE /identity/v3/projects/34ce72f3b88e443c8f89a48d79173743 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:45.022894 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:45.035249 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac49778-b646-4d79-a45b-b60ddfdd9747 service cinder] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:45.108581 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:45.109338 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:45.109713 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:45.110086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] PATH_INFO: `/v3/users/b6e2ea8f1b914820b4f6f943801fddef` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:45.112406 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e2ea8f1b914820b4f6f943801fddef)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:45.114585 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:45.165456 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:45.190874 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e2ea8f1b914820b4f6f943801fddef'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.198577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e2ea8f1b914820b4f6f943801fddef'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.199356 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e2ea8f1b914820b4f6f943801fddef'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.199954 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e2ea8f1b914820b4f6f943801fddef'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.213971 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e2ea8f1b914820b4f6f943801fddef'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.227308 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1427/2864] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:45 2022] DELETE /identity/v3/users/b6e2ea8f1b914820b4f6f943801fddef => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:45.509599 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:45.512198 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:45.526762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f5360-12af-470f-97b0-2949c24c874a admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:45.530949 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:45.604715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:45.605283 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:45.605573 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:45.605854 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] PATH_INFO: `/v3/projects/5e6143406ca34702967cff6df0dee384` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:45.607006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6143406ca34702967cff6df0dee384)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:45.608940 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:45.656801 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6143406ca34702967cff6df0dee384', 'request_id': 'req-c661670f-f13f-42de-906b-b23be7322ecb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.665565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6143406ca34702967cff6df0dee384', 'request_id': 'req-c661670f-f13f-42de-906b-b23be7322ecb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.672602 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6143406ca34702967cff6df0dee384', 'request_id': 'req-c661670f-f13f-42de-906b-b23be7322ecb'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.689865 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] The token cache is being invalidate because project 5e6143406ca34702967cff6df0dee384 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:45.690362 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.691281 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-c661670f-f13f-42de-906b-b23be7322ecb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:45.692851 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1438/2865] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:45 2022] DELETE /identity/v3/projects/5e6143406ca34702967cff6df0dee384 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:47.941246 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:47.942690 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:47.956420 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0668d057-4dc1-4b2f-be27-a82ba8c952b7 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:48.015193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:48.016362 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:48.016855 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:48.017829 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] PATH_INFO: `/v3/users/010c2c107d26433b8be2e5a9db969f3a` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:48.019723 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=010c2c107d26433b8be2e5a9db969f3a)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:48.023023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:48.085827 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010c2c107d26433b8be2e5a9db969f3a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.093468 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010c2c107d26433b8be2e5a9db969f3a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.094158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010c2c107d26433b8be2e5a9db969f3a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.094771 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010c2c107d26433b8be2e5a9db969f3a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.106627 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010c2c107d26433b8be2e5a9db969f3a'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.119227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1428/2866] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:06:47 2022] DELETE /identity/v3/users/010c2c107d26433b8be2e5a9db969f3a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:06:48.123698 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:48.124188 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:48.124289 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:48.153044 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:06:48.154248 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:48.212018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1439/2867] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:06:48.222310 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:48.223734 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:48.251762 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:48.284114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b578f1-d1e8-4e87-a3c7-57ce7ed60a6a admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:48.287733 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:48.288380 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:48.288673 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:48.288950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:48.299389 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:48.301010 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d013b0-1b9a-4fac-a446-19cc3f071a08 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:48.310588 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1429/2868] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:06:48.592715 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:06:48.594259 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:06:48.608228 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09c6d5-3be5-48e7-9c38-12b13eb9a531 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:06:48.613737 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:06:48.614380 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:06:48.614732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:06:48.614917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] PATH_INFO: `/v3/projects/41991cdb5b2e455086e3c3699462671f` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:06:48.616065 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41991cdb5b2e455086e3c3699462671f)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:06:48.618061 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:06:48.654037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:48.668861 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41991cdb5b2e455086e3c3699462671f', 'request_id': 'req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.676458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41991cdb5b2e455086e3c3699462671f', 'request_id': 'req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.682019 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41991cdb5b2e455086e3c3699462671f', 'request_id': 'req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.693559 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] The token cache is being invalidate because project 41991cdb5b2e455086e3c3699462671f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:06:48.693869 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.694499 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:06:48.695984 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1440/2869] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:06:48 2022] DELETE /identity/v3/projects/41991cdb5b2e455086e3c3699462671f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:06:58.306319 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=15 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:06:58.690596 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:07:28.840674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:07:28.841229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:07:28.841229 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:07:28.861948 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] MFA Rules not processed for user `b9319c0e1d7d4ced94b0817272b29bb4`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:07:28.863287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:07:28.917687 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1430/2870] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:07:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:07:29.156141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:07:29.157679 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:07:29.220929 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14192-e3d5-4c7e-9c8d-98eac3ac9404 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:07:29.224837 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:07:29.225462 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:07:29.225746 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:07:29.226020 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:07:29.227777 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:07:29.238466 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:07:29.239613 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:07:29.248139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1441/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:07:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:07:38.910740 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=25 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:07:39.244295 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=26 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:03.802647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:03.804049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:03.860549 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a7249-d371-4dc2-83d3-61bc7b015ffb None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:03.864158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:03.864797 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:03.865082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:03.865357 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:03.867353 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:03.928067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:03.929536 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd97a3-973a-48da-9a23-9a2aa1f98482 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:03.939142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1431/2872] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:08:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:08:09.645663 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:09.648084 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:09.725442 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f092f-ae36-4fb6-838f-b4aa5d3087cb service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:09.730458 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:09.731076 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:09.731361 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:09.731651 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:09.733506 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:09.745619 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:09.747195 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:09.757220 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1442/2873] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:08:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:08:13.934944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=48 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:13.949512 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:13.949794 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:13.950071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:13.969218 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] MFA Rules not processed for user `3f4c9c7f2ddd4403b13696794dfaab5d`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:08:13.970418 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:13.977230 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1432/2874] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:08:19.752741 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=32 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:23.963697 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:38.508066 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:38.518489 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18b905-daa6-4d0b-aeea-33f752964e77 service placement] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:38.522112 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:38.522727 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:38.523007 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:38.523280 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:38.525026 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:38.593554 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:38.595034 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:38.604891 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1443/2875] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:08:40.670473 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:40.686170 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9633d-b5e6-4c70-9dac-a1fbda746009 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:40.691451 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:40.692078 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:40.692386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:40.692662 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:40.695667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:40.709553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:40.710781 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:40.722991 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1433/2876] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:08:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:08:40.766346 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:40.768423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:40.847845 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbe841-272d-452e-bcfa-a3859792be06 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:40.852648 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:40.853305 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:40.854375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:40.854375 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] PATH_INFO: `/v3/users/b9319c0e1d7d4ced94b0817272b29bb4` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:40.855313 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9319c0e1d7d4ced94b0817272b29bb4)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:40.857325 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:40.917329 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9319c0e1d7d4ced94b0817272b29bb4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:40.918018 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9319c0e1d7d4ced94b0817272b29bb4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:40.927827 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9319c0e1d7d4ced94b0817272b29bb4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:40.936831 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9319c0e1d7d4ced94b0817272b29bb4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:40.937503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-20097623-f157-48bb-a265-92c3fffe43b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9319c0e1d7d4ced94b0817272b29bb4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:40.960732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1444/2877] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:08:40 2022] DELETE /identity/v3/users/b9319c0e1d7d4ced94b0817272b29bb4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:08:41.326505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.339550 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b018eaf-6140-45ec-b14c-cbcc8e91fc63 service placement] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.421546 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.422243 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.422784 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.423068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] PATH_INFO: `/v3/projects/d142eaffafa84ba89b53b2e8ab43639f` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.424263 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d142eaffafa84ba89b53b2e8ab43639f)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.426296 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.482177 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd142eaffafa84ba89b53b2e8ab43639f', 'request_id': 'req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:41.492164 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd142eaffafa84ba89b53b2e8ab43639f', 'request_id': 'req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:41.499517 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd142eaffafa84ba89b53b2e8ab43639f', 'request_id': 'req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:41.511071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] The token cache is being invalidate because project d142eaffafa84ba89b53b2e8ab43639f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:08:41.511436 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:41.512142 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-ace0f68d-8d99-4aaa-9b1a-9f808298fcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:08:41.519066 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1434/2878] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:08:41 2022] DELETE /identity/v3/projects/d142eaffafa84ba89b53b2e8ab43639f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:08:41.519657 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.520016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.520397 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.542147 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:08:41.543175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:41.555517 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:41.612990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1445/2879] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:08:41.621423 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.621766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.622639 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.643023 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:08:41.644227 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:41.652117 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1435/2880] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:08:41.656636 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.666548 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9468a2-8c16-48f5-ab9e-649d7e36e3f0 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.672421 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.673293 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.673723 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.674008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.674828 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=86873) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 21:08:41.675141 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.676816 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.689631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1446/2881] 10.222.0.52 () {58 vars in 1153 bytes} [Thu Aug 4 21:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 21:08:41.694020 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.705028 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3baccd-d14a-42fd-ad27-99e41f6e76df None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.710275 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.710898 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.711180 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.711454 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] PATH_INFO: `/v3/projects` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.712123 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.714086 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.751643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1436/2882] 10.222.0.52 () {60 vars in 1153 bytes} [Thu Aug 4 21:08:41 2022] POST /identity/v3/projects => generated 389 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 21:08:41.755422 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.766503 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0addfb-d99a-4532-8a9e-a28bc89ec1fe admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.830427 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.831359 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.831750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.832174 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] PATH_INFO: `/v3/users` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.832978 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.834996 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.859674 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1447/2883] 10.222.0.52 () {60 vars in 1144 bytes} [Thu Aug 4 21:08:41 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 21:08:41.863802 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.876071 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34158598-f9b2-4558-9cc0-8b047c8aa39b admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.883073 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.883975 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.884528 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.884950 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] PATH_INFO: `/v3/roles` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.885999 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.889153 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.900963 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1437/2884] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:08:41.906604 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.918603 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7d462-0776-4026-b4e9-4c36d39986bb admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.930232 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.932144 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.932483 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.933392 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] PATH_INFO: `/v3/roles` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.937271 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.939006 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e2023-0d6d-4647-9cb3-77698d7675cb admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:41.952128 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1448/2885] 10.222.0.52 () {58 vars in 1122 bytes} [Thu Aug 4 21:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 21:08:41.956283 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:41.967175 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09291145-9794-400e-96fe-95d1a5389722 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:41.972039 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:41.972805 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] REQUEST_METHOD: `PUT` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:41.973069 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:41.973481 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] PATH_INFO: `/v3/projects/781c237a5f9a47ce841cbbb1f6c05429/users/d6511d916ff44f5594ca8dc1d121cc8a/roles/5aa6a5f83ae944869649edec1f3154eb` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:41.990750 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781c237a5f9a47ce841cbbb1f6c05429, user_id=d6511d916ff44f5594ca8dc1d121cc8a, role_id=5aa6a5f83ae944869649edec1f3154eb)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:41.993515 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:42.016604 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1438/2886] 10.222.0.52 () {60 vars in 1483 bytes} [Thu Aug 4 21:08:41 2022] PUT /identity/v3/projects/781c237a5f9a47ce841cbbb1f6c05429/users/d6511d916ff44f5594ca8dc1d121cc8a/roles/5aa6a5f83ae944869649edec1f3154eb => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 21:08:42.084253 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] REQUEST_METHOD: `POST` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:42.084544 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:42.084829 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:42.116045 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.auth.core [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] MFA Rules not processed for user `d6511d916ff44f5594ca8dc1d121cc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=86873) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:08:42.117328 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:42.189037 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1449/2887] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:08:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 21:08:42.201280 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:08:42.203017 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:08:42.219276 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:42.265561 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699f512-79a1-451d-ac8e-6ccddffe8f14 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:08:42.269198 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '49153b6e04e24c8d93d323d0a4757684', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:08:42.269821 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:08:42.270096 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:08:42.270367 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:08:42.279385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:08:42.281082 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe61d93-bd87-4900-86b7-154c8e2b3502 service cinder] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:08:42.290605 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1439/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:08:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:08:52.185711 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:08:52.285834 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=20 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:09:05.799569 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:09:05.872042 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842eca3-4940-45b7-a539-4e0b5864d419 None None] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:09:05.876921 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:09:05.877611 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:09:05.877903 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:09:05.878821 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] PATH_INFO: `/v3/users/d6511d916ff44f5594ca8dc1d121cc8a` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:09:05.880100 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6511d916ff44f5594ca8dc1d121cc8a)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:09:05.882139 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:09:05.931311 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6511d916ff44f5594ca8dc1d121cc8a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:05.931958 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6511d916ff44f5594ca8dc1d121cc8a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:05.939285 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6511d916ff44f5594ca8dc1d121cc8a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:05.947310 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6511d916ff44f5594ca8dc1d121cc8a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:05.947990 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6511d916ff44f5594ca8dc1d121cc8a'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:05.966025 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1450/2889] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:09:05 2022] DELETE /identity/v3/users/d6511d916ff44f5594ca8dc1d121cc8a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:09:05.970743 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] REQUEST_METHOD: `POST` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:09:05.971285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:09:05.971285 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:09:05.994004 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.auth.core [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] MFA Rules not processed for user `477c6d25e4b7443fa6da61ccbe471661`. Rule list: `[]` (Enabled: `True`). {{(pid=86872) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 21:09:05.995206 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:09:06.058467 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1440/2890] 10.222.0.52 () {58 vars in 958 bytes} [Thu Aug 4 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 21:09:06.069836 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:09:06.071350 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:09:06.133661 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a359ea93-1215-46fe-a4fe-e3257d3de1da admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:09:06.137292 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:09:06.137917 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:09:06.138199 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:09:06.138475 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:09:06.152963 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:09:06.154398 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:09:06.165254 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1451/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:09:06.454642 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:09:06.457005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:09:06.473708 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0b151-df18-4661-a927-b2905772bac3 None None] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:09:06.478618 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:09:06.479402 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:09:06.479801 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:09:06.480083 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] PATH_INFO: `/v3/projects/781c237a5f9a47ce841cbbb1f6c05429` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:09:06.481274 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781c237a5f9a47ce841cbbb1f6c05429)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:09:06.483193 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:09:06.533068 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781c237a5f9a47ce841cbbb1f6c05429', 'request_id': 'req-7b2bc081-dbca-4203-9e6c-47a2c988b714'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:06.541461 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781c237a5f9a47ce841cbbb1f6c05429', 'request_id': 'req-7b2bc081-dbca-4203-9e6c-47a2c988b714'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:06.547084 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781c237a5f9a47ce841cbbb1f6c05429', 'request_id': 'req-7b2bc081-dbca-4203-9e6c-47a2c988b714'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:06.559615 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] The token cache is being invalidate because project 781c237a5f9a47ce841cbbb1f6c05429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86872) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:09:06.559912 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:06.560765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:09:06.562322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1441/2892] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:09:06 2022] DELETE /identity/v3/projects/781c237a5f9a47ce841cbbb1f6c05429 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:09:16.160839 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:09:16.556541 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:10:44.753997 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:10:44.762972 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1afded5-d307-4070-82f1-784dd0c6f232 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:10:44.812424 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:10:44.813049 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:10:44.813331 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:10:44.813605 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:10:44.815358 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:44.825175 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:10:44.826586 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:10:44.834762 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1452/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:10:54.831016 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=26 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:10:56.617370 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:10:56.618837 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:56.632114 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bc081-dbca-4203-9e6c-47a2c988b714 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:10:56.636590 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:10:56.637210 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:10:56.637494 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:10:56.637832 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:10:56.639586 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:56.718990 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:10:56.720505 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:10:56.732570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1442/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:10:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:10:59.188326 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:10:59.190750 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:59.191037 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:10:59.201711 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c668f-d40b-4f76-b097-7f7db1b87944 service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:10:59.205577 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:10:59.206266 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:10:59.206592 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:10:59.207674 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:10:59.209201 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca4321-f285-4e34-91a3-a0cf90c6b965 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:10:59.209888 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:59.216039 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:10:59.216768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:10:59.217615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:10:59.217615 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:10:59.219730 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:10:59.230612 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:10:59.232065 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:10:59.234423 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:10:59.236194 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:10:59.249427 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1443/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 21:10:59.249803 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1453/2896] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:11:03.417916 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:03.430493 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5e70c-b74a-442a-b8fb-ad5969bbc2de service neutron] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:03.436383 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02a04385570944ecbcaf96003f5c5753', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:03.437008 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:03.437287 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:03.437553 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:03.439554 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:03.452973 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:03.454766 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:03.466825 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1444/2897] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:11:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:11:08.902794 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:08.904205 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:08.916008 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59648e5d-c613-483b-ba99-dda09a4e5181 service nova] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:08.919901 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ef5ca1eeba741259394f0ab37dd3751', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:08.920551 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:08.920834 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:08.921105 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:08.923149 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:08.986212 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:08.987639 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:08.997335 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1454/2898] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:11:13.462140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=46 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:11:15.147385 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:15.149158 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:15.165258 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292b1be-249f-49b1-a66b-d5a3eeae0d43 service nova] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:15.170570 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:15.171226 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:15.171507 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:15.171765 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] PATH_INFO: `/v3/users/3f4c9c7f2ddd4403b13696794dfaab5d` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:15.175575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f4c9c7f2ddd4403b13696794dfaab5d)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:15.175575 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:15.223995 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f4c9c7f2ddd4403b13696794dfaab5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.231386 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f4c9c7f2ddd4403b13696794dfaab5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.232013 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f4c9c7f2ddd4403b13696794dfaab5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.232699 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f4c9c7f2ddd4403b13696794dfaab5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.245421 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f4c9c7f2ddd4403b13696794dfaab5d'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.257647 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1445/2899] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:11:15 2022] DELETE /identity/v3/users/3f4c9c7f2ddd4403b13696794dfaab5d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:11:15.435992 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:15.446454 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb50e45-210b-4521-b06c-10bf913a4ab3 service neutron] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:15.507768 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:15.508327 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:15.508631 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:15.508907 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] PATH_INFO: `/v3/projects/7535eb1709bc4bfe8a1e251fb0db4593` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:15.510717 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7535eb1709bc4bfe8a1e251fb0db4593)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:15.513193 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:15.560853 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7535eb1709bc4bfe8a1e251fb0db4593', 'request_id': 'req-9f9bab66-6308-4e3b-939b-2765e516edc4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.567864 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7535eb1709bc4bfe8a1e251fb0db4593', 'request_id': 'req-9f9bab66-6308-4e3b-939b-2765e516edc4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.574127 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7535eb1709bc4bfe8a1e251fb0db4593', 'request_id': 'req-9f9bab66-6308-4e3b-939b-2765e516edc4'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.585583 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] The token cache is being invalidate because project 7535eb1709bc4bfe8a1e251fb0db4593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:11:15.585871 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.586530 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.588565 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1455/2900] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:11:15 2022] DELETE /identity/v3/projects/7535eb1709bc4bfe8a1e251fb0db4593 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:11:15.592665 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:15.594049 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:15.607634 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1402b1a4-6c7e-4e78-a966-23d63ca4a6f8 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:15.663970 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:15.664643 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:15.664929 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:15.665204 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] PATH_INFO: `/v3/users/48e10d09dfc941dc93e7a38228a0114e` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:15.666944 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e10d09dfc941dc93e7a38228a0114e)` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:15.668895 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:15.715005 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e10d09dfc941dc93e7a38228a0114e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.722522 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e10d09dfc941dc93e7a38228a0114e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.723140 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e10d09dfc941dc93e7a38228a0114e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.723798 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e10d09dfc941dc93e7a38228a0114e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.737376 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.notifications [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e10d09dfc941dc93e7a38228a0114e'} {{(pid=86872) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:15.739322 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:11:15.752785 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1446/2901] 10.222.0.52 () {58 vars in 1224 bytes} [Thu Aug 4 21:11:15 2022] DELETE /identity/v3/users/48e10d09dfc941dc93e7a38228a0114e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:11:15.928786 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Authenticating user token {{(pid=86873) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:11:15.930208 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.fernet_utils [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86873) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:11:15.943434 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9bab66-6308-4e3b-939b-2765e516edc4 admin admin] Validating token access rules against request {{(pid=86873) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:11:15.951695 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:11:16.004522 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477c6d25e4b7443fa6da61ccbe471661', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6dad5af55404a81a0a649f9204e78ff', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86873) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:11:16.005393 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:11:16.005732 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:11:16.006102 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] PATH_INFO: `/v3/projects/91f8baeb2a84415f8b767cf708fcbf64` {{(pid=86873) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:11:16.007634 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f8baeb2a84415f8b767cf708fcbf64)` {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:11:16.010791 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] RBAC: Authorization granted {{(pid=86873) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:11:16.056729 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f8baeb2a84415f8b767cf708fcbf64', 'request_id': 'req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:16.064925 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f8baeb2a84415f8b767cf708fcbf64', 'request_id': 'req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:16.070654 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f8baeb2a84415f8b767cf708fcbf64', 'request_id': 'req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8'} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:16.082665 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] The token cache is being invalidate because project 91f8baeb2a84415f8b767cf708fcbf64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=86873) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 21:11:16.082948 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:16.083580 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG keystone.notifications [None req-2f6546ce-32d8-4118-a4fa-4563dd1ed0a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=86873) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 21:11:16.085103 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: [pid: 86873|app: 0|req: 1456/2902] 10.222.0.52 () {58 vars in 1233 bytes} [Thu Aug 4 21:11:15 2022] DELETE /identity/v3/projects/91f8baeb2a84415f8b767cf708fcbf64 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 21:11:25.749137 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=4 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:11:26.079775 dv-cinder-836027-4-iscsi devstack@keystone.service[86873]: DEBUG dbcounter [-] [86873] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=86873) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 21:13:43.064067 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Authenticating user token {{(pid=86872) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 21:13:43.065547 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:13:43.207441 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7aeab-07bf-4b1e-86a9-d61062335c02 admin admin] Validating token access rules against request {{(pid=86872) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 21:13:43.266915 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72012ef48a864233970d32552c403409', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '07b38a76369e4d47bac7906590383b26', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=86872) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 21:13:43.267538 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] REQUEST_METHOD: `GET` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 21:13:43.267819 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] SCRIPT_NAME: `/identity` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 21:13:43.268098 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=86872) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 21:13:43.269877 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.fernet_utils [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=86872) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 21:13:43.281463 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 21:13:43.283057 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b31f3a-1206-4d1f-ae53-ed9d39438e53 service placement] RBAC: Authorization granted {{(pid=86872) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 21:13:43.293070 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: [pid: 86872|app: 0|req: 1447/2903] 10.222.0.52 () {60 vars in 1365 bytes} [Thu Aug 4 21:13:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 21:13:53.288667 dv-cinder-836027-4-iscsi devstack@keystone.service[86872]: DEBUG dbcounter [-] [86872] Writing DB stats keystone:SELECT=89 {{(pid=86872) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}